Jul 02 16:54:25.191741 ip-10-0-129-102 systemd[1]: Starting Kubernetes Kubelet... Jul 02 16:54:25.670142 ip-10-0-129-102 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 16:54:25.670142 ip-10-0-129-102 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 16:54:25.670142 ip-10-0-129-102 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 16:54:25.670142 ip-10-0-129-102 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 16:54:25.670142 ip-10-0-129-102 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 16:54:25.670142 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.600477 2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 16:54:25.670142 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604904 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 16:54:25.670142 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604918 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 16:54:25.670142 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604922 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 16:54:25.670142 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604925 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 16:54:25.670142 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604928 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 16:54:25.670142 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604931 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 16:54:25.670142 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604934 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 16:54:25.670142 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604937 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604940 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604944 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604947 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604950 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604952 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604955 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604959 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604962 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604965 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604968 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604970 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604974 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604976 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604979 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604981 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604984 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604988 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604992 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 16:54:25.671442 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604995 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.604998 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605002 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605006 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605009 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605012 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605015 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605018 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605021 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605024 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605026 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605030 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605034 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605037 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605040 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605042 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605044 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605047 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605049 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 16:54:25.672551 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605052 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605055 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605057 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605060 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605062 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605071 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605074 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605076 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605078 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605081 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605083 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605086 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605089 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605091 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605095 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605097 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605100 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605103 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605105 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605108 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 16:54:25.673678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605110 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605113 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605116 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605118 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605120 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605123 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605127 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605130 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605133 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605136 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605138 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605141 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605144 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605147 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605150 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605152 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605155 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605158 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605161 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605164 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 16:54:25.674564 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605166 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605169 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605172 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605174 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605177 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605179 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605182 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605185 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605187 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605190 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605192 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605195 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605198 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605201 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605948 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605955 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605959 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605962 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605965 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605968 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 16:54:25.675459 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605971 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605974 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605977 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605980 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605982 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605985 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605988 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605991 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605993 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605996 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.605999 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606001 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606004 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606006 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606009 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606011 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606014 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606016 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606019 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606022 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 16:54:25.676557 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606024 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606027 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606029 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606032 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606035 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606038 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606040 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606043 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606046 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606049 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606052 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606054 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606057 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606060 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606063 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606067 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606070 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606073 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606076 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 16:54:25.677464 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606078 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606081 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606083 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606086 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606089 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606091 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606094 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606097 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606099 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606102 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606104 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606107 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606109 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606112 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606114 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606117 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606119 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606122 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606124 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606128 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 16:54:25.678579 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606132 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 16:54:25.678666 ip-10-0-129-102 systemd[1]: Started Kubernetes Kubelet. Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606135 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606138 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606141 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606143 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606146 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606150 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606152 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606155 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606157 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606160 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606162 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606165 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606168 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606170 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606173 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606175 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606178 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606181 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 16:54:25.679629 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606183 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606188 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606191 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606194 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606197 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606200 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606202 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606206 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606209 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606211 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606214 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606217 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606219 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606222 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606225 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606612 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606617 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606621 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606626 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 16:54:25.680848 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606630 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606634 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606636 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606639 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606642 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606645 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606647 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606650 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606652 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606655 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606657 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606660 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606662 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606667 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606670 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606672 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606675 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606678 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606680 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606682 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 16:54:25.681945 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606685 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606688 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606691 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606693 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606696 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606698 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606701 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606704 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606707 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606709 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606711 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606714 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606717 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606720 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606722 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606725 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606728 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606730 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606732 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606735 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 16:54:25.683074 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606737 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606740 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606742 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606745 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606747 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606751 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606755 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606758 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606761 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606765 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606767 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606770 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606772 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606775 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606778 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606780 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606783 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606786 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606788 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 16:54:25.684133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606791 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606793 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606796 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606798 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606801 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606803 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606806 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606813 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606816 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606818 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606821 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606823 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606826 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606828 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606831 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606833 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606836 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606838 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606842 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606846 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 16:54:25.685166 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606849 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606851 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606854 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606856 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606859 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606861 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606864 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606867 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606870 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606873 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606875 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606877 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606880 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606882 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606885 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.606889 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.606960 2556 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.606967 2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.606974 2556 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.606978 2556 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.606983 2556 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 16:54:25.686061 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.606986 2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.606990 2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.606994 2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.606997 2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.607000 2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.607004 2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.607010 2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.607015 2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.607017 2556 flags.go:64] FLAG: --cgroup-root="" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.607022 2556 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.607027 2556 flags.go:64] FLAG: --client-ca-file="" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.607030 2556 flags.go:64] FLAG: --cloud-provider="external" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.607033 2556 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608049 2556 flags.go:64] FLAG: --cluster-domain="" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608053 2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608057 2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608060 2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608063 2556 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608068 2556 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608071 2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608074 2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608077 2556 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608080 2556 flags.go:64] FLAG: --contention-profiling="false" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608083 2556 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 16:54:25.686951 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608086 2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608090 2556 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608093 2556 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608097 2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608100 2556 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608103 2556 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608106 2556 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608109 2556 flags.go:64] FLAG: --enable-server="true" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608113 2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608118 2556 flags.go:64] FLAG: --event-burst="100" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608121 2556 flags.go:64] FLAG: --event-qps="50" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608124 2556 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608127 2556 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608130 2556 flags.go:64] FLAG: --eviction-hard="" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608134 2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608137 2556 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608140 2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608145 2556 flags.go:64] FLAG: --eviction-soft="" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608149 2556 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608152 2556 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608155 2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608158 2556 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608161 2556 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608164 2556 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608167 2556 flags.go:64] FLAG: --feature-gates="" Jul 02 16:54:25.688060 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608170 2556 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608174 2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608177 2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608180 2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608183 2556 flags.go:64] FLAG: --healthz-port="10248" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608185 2556 flags.go:64] FLAG: --help="false" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608188 2556 flags.go:64] FLAG: --hostname-override="ip-10-0-129-102.ec2.internal" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608191 2556 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608194 2556 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608197 2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608200 2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608203 2556 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608206 2556 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608209 2556 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608212 2556 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608215 2556 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608218 2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608221 2556 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608224 2556 flags.go:64] FLAG: --kube-reserved="" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608227 2556 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608229 2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608232 2556 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608235 2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608238 2556 flags.go:64] FLAG: --lock-file="" Jul 02 16:54:25.689163 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608245 2556 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608249 2556 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608252 2556 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608257 2556 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608260 2556 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608263 2556 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608277 2556 flags.go:64] FLAG: --logging-format="text" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608280 2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608283 2556 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608286 2556 flags.go:64] FLAG: --manifest-url="" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608289 2556 flags.go:64] FLAG: --manifest-url-header="" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608293 2556 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608296 2556 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608300 2556 flags.go:64] FLAG: --max-pods="110" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608303 2556 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608307 2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608310 2556 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608313 2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608316 2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608318 2556 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608321 2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608330 2556 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608333 2556 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608336 2556 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 16:54:25.693629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608339 2556 flags.go:64] FLAG: --pod-cidr="" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608343 2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608348 2556 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608351 2556 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608354 2556 flags.go:64] FLAG: --pods-per-core="0" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608357 2556 flags.go:64] FLAG: --port="10250" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608360 2556 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608362 2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a475ca703ee6ee63" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608367 2556 flags.go:64] FLAG: --qos-reserved="" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608370 2556 flags.go:64] FLAG: --read-only-port="10255" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608375 2556 flags.go:64] FLAG: --register-node="true" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608378 2556 flags.go:64] FLAG: --register-with-taints="" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608382 2556 flags.go:64] FLAG: --registry-burst="10" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608385 2556 flags.go:64] FLAG: --registry-qps="5" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608388 2556 flags.go:64] FLAG: --reserved-cpus="" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608391 2556 flags.go:64] FLAG: --reserved-memory="" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608395 2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608398 2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608401 2556 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608404 2556 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608407 2556 flags.go:64] FLAG: --runonce="false" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608410 2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608413 2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608415 2556 flags.go:64] FLAG: --seccomp-default="false" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608418 2556 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 16:54:25.694746 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608421 2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608424 2556 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608427 2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608430 2556 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608433 2556 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608435 2556 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608438 2556 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608441 2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608444 2556 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608448 2556 flags.go:64] FLAG: --system-cgroups="" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608451 2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608456 2556 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608459 2556 flags.go:64] FLAG: --tls-cert-file="" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608462 2556 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608466 2556 flags.go:64] FLAG: --tls-min-version="" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608470 2556 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608473 2556 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608477 2556 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608480 2556 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608483 2556 flags.go:64] FLAG: --v="2" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608487 2556 flags.go:64] FLAG: --version="false" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608491 2556 flags.go:64] FLAG: --vmodule="" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608495 2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608498 2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608616 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 16:54:27.207369 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608620 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608623 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608626 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608629 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608631 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608634 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608636 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608639 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608641 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608644 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608646 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608649 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608651 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608654 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608656 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608659 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608662 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608665 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608667 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608670 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 16:54:27.226133 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608672 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608675 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608678 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608681 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608686 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608689 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608693 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608696 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608699 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608702 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608705 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608708 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608711 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608714 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608717 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608719 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608722 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608725 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608727 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 16:54:27.227044 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608730 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608732 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608735 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608737 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608740 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608743 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608745 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608748 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608750 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608753 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608755 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608759 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608761 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608763 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608766 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608769 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608772 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608776 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608778 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608781 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 16:54:27.227811 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608783 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608786 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608789 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608791 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608794 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608797 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608799 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608802 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608804 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608807 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608809 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608812 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608814 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608818 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608821 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608824 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608827 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608830 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608832 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608835 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 16:54:27.228468 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608837 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608840 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608842 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608845 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608848 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608850 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608853 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608856 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608859 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608862 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608865 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608868 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608870 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608873 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608876 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608878 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608881 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608885 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 16:54:27.229111 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.608888 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 16:54:27.229718 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.608894 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 16:54:27.229718 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.616218 2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 16:54:27.229718 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.616234 2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 16:54:27.229718 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.616289 2556 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 16:54:27.229718 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.616298 2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 16:54:27.229718 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616356 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 16:54:27.229718 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616361 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 16:54:27.229718 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616364 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 16:54:27.229718 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616367 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 16:54:27.229718 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616370 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 16:54:27.229718 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616373 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 16:54:27.229718 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616376 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 16:54:27.229718 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616379 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 16:54:27.229718 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616382 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 16:54:27.229718 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616384 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616387 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616389 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616392 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616395 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616397 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616402 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616405 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616408 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616411 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616415 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616419 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616422 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616425 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616428 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616430 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616433 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616436 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616438 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 16:54:27.230272 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616441 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616444 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616447 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616450 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616452 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616455 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616457 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616460 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616462 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616465 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616468 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616471 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616474 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616477 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616479 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616482 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616484 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616487 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616489 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 16:54:27.230815 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616492 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616495 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616498 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616502 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616505 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616507 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616510 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616512 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616515 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616517 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616520 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616522 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616525 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616527 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616530 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616533 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616537 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616540 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616543 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616546 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 16:54:27.231438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616549 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616552 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616555 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616558 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616560 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616565 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616568 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616570 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616573 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616575 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616578 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616581 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616583 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616586 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616589 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616591 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616594 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616596 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616599 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616602 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 16:54:27.231995 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616604 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 16:54:27.232689 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616607 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 16:54:27.232689 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616609 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 16:54:27.232689 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616612 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 16:54:27.232689 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616615 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 16:54:27.232689 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616617 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 16:54:27.232689 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616620 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 16:54:27.232689 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616625 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 16:54:27.232689 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616628 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 16:54:27.232689 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616631 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 16:54:27.232689 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616633 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 16:54:27.232689 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616636 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 16:54:27.232689 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.616641 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 16:54:27.232689 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616756 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 16:54:27.232689 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616761 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 16:54:27.232689 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616764 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616768 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616773 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616775 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616778 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616781 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616783 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616786 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616789 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616791 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616794 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616796 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616799 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616801 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616804 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616806 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616809 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616811 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616814 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616816 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 16:54:27.233117 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616819 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616821 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616823 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616826 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616829 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616833 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616836 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616839 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616841 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616844 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616847 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616849 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616852 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616855 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616858 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616861 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616863 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616866 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616869 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616872 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616874 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 16:54:27.233724 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616877 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616879 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616882 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616885 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616887 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616890 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616892 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616895 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616897 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616900 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616902 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616906 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616910 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616913 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616916 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616918 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616922 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616925 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616929 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 16:54:27.234312 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616931 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616934 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616936 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616939 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616941 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616944 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616948 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616950 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616953 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616955 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616958 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616960 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616963 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616965 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616968 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616970 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616973 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616976 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616979 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616981 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 16:54:27.235440 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616984 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 16:54:27.236313 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616986 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 16:54:27.236313 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616989 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 16:54:27.236313 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616991 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 16:54:27.236313 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616994 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 16:54:27.236313 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616996 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 16:54:27.236313 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.616999 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 16:54:27.236313 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.617001 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 16:54:27.236313 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.617004 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 16:54:27.236313 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.617007 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 16:54:27.236313 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.617010 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 16:54:27.236313 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.617012 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 16:54:27.236313 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.617015 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 16:54:27.236313 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.617018 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 16:54:27.236313 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.617020 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 16:54:27.236313 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.617024 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 16:54:27.236313 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:25.617028 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 16:54:27.237198 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.617033 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 16:54:27.237198 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.617139 2556 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 16:54:27.237198 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.619201 2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 16:54:27.237198 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.620593 2556 server.go:1019] "Starting client certificate rotation" Jul 02 16:54:27.237198 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.620691 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 16:54:27.237198 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.620723 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 16:54:27.237198 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.623487 2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 16:54:27.237198 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.627442 2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 16:54:27.237198 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.641133 2556 log.go:25] "Validated CRI v1 runtime API" Jul 02 16:54:27.237198 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.643976 2556 log.go:25] "Validated CRI v1 image API" Jul 02 16:54:27.237198 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.646602 2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 16:54:27.237198 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.653983 2556 fs.go:135] Filesystem UUIDs: map[14ad73b4-0dcd-4951-8549-791204aaf078:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f3b66f2e-51dc-43d0-b9c0-40ee64365da3:/dev/nvme0n1p3] Jul 02 16:54:27.237833 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.654029 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 16:54:27.237833 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.654036 2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.659290 2556 manager.go:216] Machine: {Timestamp:2026-07-02 16:54:25.657875887 +0000 UTC m=+0.347639160 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102557 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26ce598186180568f97f77420c5753 SystemUUID:ec26ce59-8186-1805-68f9-7f77420c5753 BootID:795504f5-1ad3-42af-81ad-b2d74d44c287 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4a:7e:4b:36:b7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4a:7e:4b:36:b7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:1e:eb:4d:40:d0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.659394 2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.659515 2556 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.662814 2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.662845 2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-102.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.663578 2556 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.663587 2556 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.663595 2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.664346 2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.665323 2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.668385 2556 state_mem.go:36] "Initialized new in-memory state store" Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.668521 2556 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.668550 2556 kubelet.go:486] "Attempting to sync node with API server" Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.668561 2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.668577 2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.668587 2556 kubelet.go:398] "Adding apiserver pod source" Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.668597 2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 16:54:27.240610 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.668964 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.669002 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.671368 2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.672714 2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.672750 2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.672930 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.672946 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.672955 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.672963 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.672973 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.672979 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.673663 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.673673 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.673682 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.673691 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.673703 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.673713 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.674637 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.674644 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.677152 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gz8w5" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.677986 2556 server.go:1268] "Started kubelet" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.678368 2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.704349 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gz8w5" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.703908 2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.704595 2556 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.705685 2556 server.go:310] "Adding debug handlers to kubelet server" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.706952 2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-102.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:25.707594 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:25.707596 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-102.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.708055 2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:25.712207 2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.720251 2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.720285 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:25.720479 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-102.ec2.internal\" not found" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.720850 2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.720854 2556 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.720957 2556 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.721099 2556 reconstruct.go:97] "Volume reconstruction finished" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.721107 2556 reconciler.go:29] "Reconciler: start to sync state" Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.721290 2556 factory.go:55] Registering systemd factory Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.721305 2556 factory.go:223] Registration of the systemd container factory successfully Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.721601 2556 factory.go:153] Registering CRI-O factory Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.721619 2556 factory.go:223] Registration of the crio container factory successfully Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.721738 2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.721760 2556 factory.go:103] Registering Raw factory Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.721771 2556 manager.go:1196] Started watching for new ooms in manager Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.722185 2556 manager.go:349] Starting recovery of all containers Jul 02 16:54:27.241500 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.722325 2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:25.725053 2556 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-102.ec2.internal\" not found" node="ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.735722 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.736497 2556 manager.go:354] Recovery completed Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.741190 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.746711 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasSufficientMemory" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.746740 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.746753 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasSufficientPID" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.747202 2556 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.747212 2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.747235 2556 state_mem.go:36] "Initialized new in-memory state store" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.747240 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.748039 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasSufficientMemory" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.748064 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.748074 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasSufficientPID" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.748331 2556 policy_none.go:49] "None policy: Start" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.748344 2556 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:25.748354 2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.126134 2556 policy_none.go:47] "Start" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.175870 2556 manager.go:337] "Starting Device Plugin manager" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:27.175915 2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.175929 2556 server.go:87] "Starting device plugin registration server" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.176242 2556 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.176258 2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.176615 2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.176693 2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.176703 2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:27.177226 2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:27.177279 2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-102.ec2.internal\" not found" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.183100 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.183134 2556 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.183171 2556 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:27.183244 2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.185240 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.276629 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.277683 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasSufficientMemory" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.277710 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.277719 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasSufficientPID" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.277742 2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.283931 2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-102.ec2.internal"] Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.283987 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.285503 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasSufficientMemory" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.285527 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.285537 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasSufficientPID" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.286703 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.286846 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.286873 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 16:54:27.377961 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.286876 2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:27.286898 2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-102.ec2.internal\": node \"ip-10-0-129-102.ec2.internal\" not found" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.288617 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasSufficientMemory" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.288642 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.288651 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasSufficientPID" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.288618 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasSufficientMemory" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.288705 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.288717 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasSufficientPID" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.289861 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.289883 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.292944 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasSufficientMemory" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.292968 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.292978 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeHasSufficientPID" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.299298 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5353c433e8c4b5cfc4a4ccb34b7b3e4d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal\" (UID: \"5353c433e8c4b5cfc4a4ccb34b7b3e4d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.299321 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5353c433e8c4b5cfc4a4ccb34b7b3e4d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal\" (UID: \"5353c433e8c4b5cfc4a4ccb34b7b3e4d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.299337 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b66c94e3b0824384de48db496e306579-config\") pod \"kube-apiserver-proxy-ip-10-0-129-102.ec2.internal\" (UID: \"b66c94e3b0824384de48db496e306579\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:27.300183 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-102.ec2.internal\" not found" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:27.320933 2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-102.ec2.internal\" not found" node="ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.380215 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:27.325463 2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-102.ec2.internal\" not found" node="ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.558750 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:27.400356 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-102.ec2.internal\" not found" Jul 02 16:54:27.558750 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.400363 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5353c433e8c4b5cfc4a4ccb34b7b3e4d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal\" (UID: \"5353c433e8c4b5cfc4a4ccb34b7b3e4d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.558750 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.400400 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5353c433e8c4b5cfc4a4ccb34b7b3e4d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal\" (UID: \"5353c433e8c4b5cfc4a4ccb34b7b3e4d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.558750 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.400418 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b66c94e3b0824384de48db496e306579-config\") pod \"kube-apiserver-proxy-ip-10-0-129-102.ec2.internal\" (UID: \"b66c94e3b0824384de48db496e306579\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.558750 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.400425 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5353c433e8c4b5cfc4a4ccb34b7b3e4d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal\" (UID: \"5353c433e8c4b5cfc4a4ccb34b7b3e4d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.558750 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.400400 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5353c433e8c4b5cfc4a4ccb34b7b3e4d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal\" (UID: \"5353c433e8c4b5cfc4a4ccb34b7b3e4d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.558750 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.400460 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b66c94e3b0824384de48db496e306579-config\") pod \"kube-apiserver-proxy-ip-10-0-129-102.ec2.internal\" (UID: \"b66c94e3b0824384de48db496e306579\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.558750 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:27.500965 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-102.ec2.internal\" not found" Jul 02 16:54:27.729426 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:27.601482 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-102.ec2.internal\" not found" Jul 02 16:54:27.729426 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.625656 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.729426 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.627810 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-102.ec2.internal" Jul 02 16:54:27.729426 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:27.702016 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-102.ec2.internal\" not found" Jul 02 16:54:27.952417 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:27.802338 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-102.ec2.internal\" not found" Jul 02 16:54:27.952417 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:27.903129 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-102.ec2.internal\" not found" Jul 02 16:54:28.166093 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.996979 2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 16:54:28.166093 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.997127 2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 16:54:28.166093 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:27.997151 2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 16:54:28.166093 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:28.004396 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-102.ec2.internal\" not found" Jul 02 16:54:28.166093 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:28.088532 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 16:49:27 +0000 UTC" deadline="2028-01-15 02:12:11.79008192 +0000 UTC" Jul 02 16:54:28.166093 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:28.088718 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13473h17m43.701369706s" Jul 02 16:54:28.166093 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:28.104650 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-102.ec2.internal\" not found" Jul 02 16:54:28.369000 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:28.205196 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-102.ec2.internal\" not found" Jul 02 16:54:28.369000 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:28.241174 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb66c94e3b0824384de48db496e306579.slice/crio-38a92cc6d9739accfedc2a8480148d7fa981377c22c85174673945f4eb4fbde3 WatchSource:0}: Error finding container 38a92cc6d9739accfedc2a8480148d7fa981377c22c85174673945f4eb4fbde3: Status 404 returned error can't find the container with id 38a92cc6d9739accfedc2a8480148d7fa981377c22c85174673945f4eb4fbde3 Jul 02 16:54:28.369000 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:28.241489 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5353c433e8c4b5cfc4a4ccb34b7b3e4d.slice/crio-1a7506ded6c4703750617fa7f6edea789c4bece42ae4902afa0877871217e136 WatchSource:0}: Error finding container 1a7506ded6c4703750617fa7f6edea789c4bece42ae4902afa0877871217e136: Status 404 returned error can't find the container with id 1a7506ded6c4703750617fa7f6edea789c4bece42ae4902afa0877871217e136 Jul 02 16:54:28.369000 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:28.246625 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 16:54:28.369000 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:28.293958 2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 16:54:28.369000 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:28.306175 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-102.ec2.internal\" not found" Jul 02 16:54:28.609426 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:28.406576 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-102.ec2.internal\" not found" Jul 02 16:54:28.609426 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:28.446318 2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 16:54:28.609426 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:28.498232 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal" Jul 02 16:54:28.609426 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:28.508285 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 16:54:28.609426 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:28.509249 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-102.ec2.internal" Jul 02 16:54:28.609426 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:28.516902 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 16:54:28.856605 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:28.841265 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 16:54:29.064033 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.047449 2556 apiserver.go:52] "Watching apiserver" Jul 02 16:54:29.064033 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.060023 2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 16:54:29.064033 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.061118 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-wbgfs","openshift-cluster-node-tuning-operator/tuned-ls7bj","openshift-image-registry/node-ca-clww4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal","openshift-multus/multus-4mb2b","openshift-multus/multus-additional-cni-plugins-fnsfd","openshift-network-operator/iptables-alerter-cxfrf","kube-system/global-pull-secret-syncer-76pzw","kube-system/kube-apiserver-proxy-ip-10-0-129-102.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t","openshift-dns/node-resolver-6c5gj","openshift-multus/network-metrics-daemon-bzrql","openshift-network-diagnostics/network-check-target-5xcjz","openshift-ovn-kubernetes/ovnkube-node-qx6zp"] Jul 02 16:54:29.064033 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.063951 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.067477 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.066801 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 16:54:29.067477 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.067043 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 16:54:29.067477 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.067297 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:29.067477 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.067373 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:54:29.067477 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.067466 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 16:54:29.067477 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.067475 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-d6c48\"" Jul 02 16:54:29.069900 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.069875 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-clww4" Jul 02 16:54:29.071907 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.071885 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.072840 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.072615 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 16:54:29.072840 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.072624 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 16:54:29.072840 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.072698 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hr462\"" Jul 02 16:54:29.072840 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.072733 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 16:54:29.074645 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.074111 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.075062 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.074979 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 16:54:29.075170 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.075062 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 16:54:29.075957 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.075799 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 16:54:29.075957 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.075869 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 16:54:29.076199 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.076136 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kqzbr\"" Jul 02 16:54:29.077041 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.076368 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 16:54:29.077041 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.076434 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.077041 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.076725 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 16:54:29.077041 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.076878 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 16:54:29.077041 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.076919 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jh742\"" Jul 02 16:54:29.077757 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.077518 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 16:54:29.077757 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.077612 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 16:54:29.082262 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.078626 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 16:54:29.082262 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.078954 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 16:54:29.082262 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.079928 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pqpd2\"" Jul 02 16:54:29.082484 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.082273 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 16:54:29.085398 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.085373 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:29.085485 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.085461 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:29.087767 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.087749 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cxfrf" Jul 02 16:54:29.090274 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.090076 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 16:54:29.090375 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.090363 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 16:54:29.090431 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.090388 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9bjq9\"" Jul 02 16:54:29.090568 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.090551 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 16:54:29.092183 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.091643 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.096011 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.095722 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 16:54:29.097136 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.096995 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 16:54:29.097136 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.097076 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mqr9x\"" Jul 02 16:54:29.098197 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.097699 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 16:54:29.098289 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.098251 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wbgfs" Jul 02 16:54:29.100752 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.100545 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 16:54:29.100752 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.100584 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-p8lwb\"" Jul 02 16:54:29.100913 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.100892 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 16:54:29.101085 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.101065 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6c5gj" Jul 02 16:54:29.103673 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.103651 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:29.103748 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.103721 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:54:29.103802 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.103754 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 16:54:29.103923 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.103903 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-k6jgw\"" Jul 02 16:54:29.104019 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.103911 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 16:54:29.106901 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.106879 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 16:54:29.110736 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.110713 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-slash\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.110844 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.110751 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-run-ovn-kubernetes\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.110844 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.110779 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-run-netns\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.110844 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.110819 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-device-dir\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.110985 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.110901 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h9cc\" (UniqueName: \"kubernetes.io/projected/30310ad4-f069-4707-bef7-47f64a734db4-kube-api-access-9h9cc\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.110985 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.110935 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-env-overrides\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.110985 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.110963 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-ovnkube-script-lib\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.111133 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.110992 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28783fce-657d-4461-84ca-ae848eb09b15-tmp-dir\") pod \"node-resolver-6c5gj\" (UID: \"28783fce-657d-4461-84ca-ae848eb09b15\") " pod="openshift-dns/node-resolver-6c5gj" Jul 02 16:54:29.111133 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111016 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cce0691f-7213-4afa-b73e-d22536b1861a-cni-binary-copy\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.111133 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111040 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-sysctl-conf\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.111133 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111074 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-systemd-units\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.111133 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111097 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-host\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.111133 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111129 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-run-netns\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.111443 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111196 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3a271183-1ef5-49c2-a88f-7ea46b2b8864-kubelet-config\") pod \"global-pull-secret-syncer-76pzw\" (UID: \"3a271183-1ef5-49c2-a88f-7ea46b2b8864\") " pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:29.111443 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111224 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-run-openvswitch\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.111443 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111250 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-ovnkube-config\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.111443 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111285 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/53d27a00-961d-4ea1-9312-39cfc71e8bb5-agent-certs\") pod \"konnectivity-agent-wbgfs\" (UID: \"53d27a00-961d-4ea1-9312-39cfc71e8bb5\") " pod="kube-system/konnectivity-agent-wbgfs" Jul 02 16:54:29.111443 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111308 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-cnibin\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.111443 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111331 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-os-release\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.111443 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111353 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-var-lib-kubelet\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.111443 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111378 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3a271183-1ef5-49c2-a88f-7ea46b2b8864-dbus\") pod \"global-pull-secret-syncer-76pzw\" (UID: \"3a271183-1ef5-49c2-a88f-7ea46b2b8864\") " pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:29.111443 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111402 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eca9108d-442e-458e-80cb-8a5f5e91885a-tmp\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.111443 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111433 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/98403e6f-9dcd-43ef-83f5-7b04bd39f933-iptables-alerter-script\") pod \"iptables-alerter-cxfrf\" (UID: \"98403e6f-9dcd-43ef-83f5-7b04bd39f933\") " pod="openshift-network-operator/iptables-alerter-cxfrf" Jul 02 16:54:29.111877 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111470 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/53d27a00-961d-4ea1-9312-39cfc71e8bb5-konnectivity-ca\") pod \"konnectivity-agent-wbgfs\" (UID: \"53d27a00-961d-4ea1-9312-39cfc71e8bb5\") " pod="kube-system/konnectivity-agent-wbgfs" Jul 02 16:54:29.111877 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111537 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfdzd\" (UniqueName: \"kubernetes.io/projected/dfb32fb2-9a93-4405-be31-8c26f66032e1-kube-api-access-rfdzd\") pod \"node-ca-clww4\" (UID: \"dfb32fb2-9a93-4405-be31-8c26f66032e1\") " pod="openshift-image-registry/node-ca-clww4" Jul 02 16:54:29.111877 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111576 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-var-lib-cni-multus\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.111877 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111602 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret\") pod \"global-pull-secret-syncer-76pzw\" (UID: \"3a271183-1ef5-49c2-a88f-7ea46b2b8864\") " pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:29.111877 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111628 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-sysctl-d\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.111877 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111683 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n78qs\" (UniqueName: \"kubernetes.io/projected/eca9108d-442e-458e-80cb-8a5f5e91885a-kube-api-access-n78qs\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.111877 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111726 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cce0691f-7213-4afa-b73e-d22536b1861a-multus-daemon-config\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.111877 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111750 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-modprobe-d\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.111877 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111796 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-kubernetes\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.111877 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111827 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-etc-selinux\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.111877 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111855 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-var-lib-openvswitch\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.111877 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111879 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-cni-bin\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.112394 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111903 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa88be7d-b349-4071-8f78-2cdafc2fd875-cnibin\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.112394 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111940 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-registration-dir\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.112394 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111963 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa88be7d-b349-4071-8f78-2cdafc2fd875-system-cni-dir\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.112394 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.111987 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5cf2\" (UniqueName: \"kubernetes.io/projected/aa88be7d-b349-4071-8f78-2cdafc2fd875-kube-api-access-v5cf2\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.112394 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.112015 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-multus-conf-dir\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.112394 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.112040 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wst2s\" (UniqueName: \"kubernetes.io/projected/cce0691f-7213-4afa-b73e-d22536b1861a-kube-api-access-wst2s\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.112394 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.112063 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-sysconfig\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.112394 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.112095 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-systemd\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.112394 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.112132 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-kubelet\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.112394 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.112175 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-etc-kubernetes\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.112851 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.112775 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-sys-fs\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.112851 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.112816 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-run-systemd\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.112851 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.112842 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-run-ovn\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.113001 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.112867 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-ovn-node-metrics-cert\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.113001 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.112932 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aa88be7d-b349-4071-8f78-2cdafc2fd875-os-release\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.113001 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.112959 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa88be7d-b349-4071-8f78-2cdafc2fd875-cni-binary-copy\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.113001 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.112985 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-multus-cni-dir\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.113202 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113010 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-lib-modules\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.113202 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113050 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-log-socket\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.113202 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113077 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbnlx\" (UniqueName: \"kubernetes.io/projected/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-kube-api-access-vbnlx\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.113202 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113109 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aa88be7d-b349-4071-8f78-2cdafc2fd875-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.113202 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113134 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz5xl\" (UniqueName: \"kubernetes.io/projected/28783fce-657d-4461-84ca-ae848eb09b15-kube-api-access-vz5xl\") pod \"node-resolver-6c5gj\" (UID: \"28783fce-657d-4461-84ca-ae848eb09b15\") " pod="openshift-dns/node-resolver-6c5gj" Jul 02 16:54:29.113202 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113182 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-tuned\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.113467 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113208 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.113467 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113231 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-socket-dir\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.113467 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113262 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-cni-netd\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.113467 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113286 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.113467 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113312 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa88be7d-b349-4071-8f78-2cdafc2fd875-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.113467 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113337 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aa88be7d-b349-4071-8f78-2cdafc2fd875-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.113467 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113361 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dfb32fb2-9a93-4405-be31-8c26f66032e1-serviceca\") pod \"node-ca-clww4\" (UID: \"dfb32fb2-9a93-4405-be31-8c26f66032e1\") " pod="openshift-image-registry/node-ca-clww4" Jul 02 16:54:29.113467 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113403 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-hostroot\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.113467 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113437 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-sys\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.113467 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113463 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-node-log\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.113910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113493 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98403e6f-9dcd-43ef-83f5-7b04bd39f933-host-slash\") pod \"iptables-alerter-cxfrf\" (UID: \"98403e6f-9dcd-43ef-83f5-7b04bd39f933\") " pod="openshift-network-operator/iptables-alerter-cxfrf" Jul 02 16:54:29.113910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113517 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/28783fce-657d-4461-84ca-ae848eb09b15-hosts-file\") pod \"node-resolver-6c5gj\" (UID: \"28783fce-657d-4461-84ca-ae848eb09b15\") " pod="openshift-dns/node-resolver-6c5gj" Jul 02 16:54:29.113910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113539 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntfj8\" (UniqueName: \"kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8\") pod \"network-check-target-5xcjz\" (UID: \"3e206cf7-e305-45d7-be70-8b153b93bffa\") " pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:29.113910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113561 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dfb32fb2-9a93-4405-be31-8c26f66032e1-host\") pod \"node-ca-clww4\" (UID: \"dfb32fb2-9a93-4405-be31-8c26f66032e1\") " pod="openshift-image-registry/node-ca-clww4" Jul 02 16:54:29.113910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113582 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-run-k8s-cni-cncf-io\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.113910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113607 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-run\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.113910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113630 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-var-lib-kubelet\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.113910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113653 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-etc-openvswitch\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.113910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113678 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkfhj\" (UniqueName: \"kubernetes.io/projected/98403e6f-9dcd-43ef-83f5-7b04bd39f933-kube-api-access-wkfhj\") pod \"iptables-alerter-cxfrf\" (UID: \"98403e6f-9dcd-43ef-83f5-7b04bd39f933\") " pod="openshift-network-operator/iptables-alerter-cxfrf" Jul 02 16:54:29.113910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113702 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-system-cni-dir\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.113910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113734 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-multus-socket-dir-parent\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.113910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113771 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-var-lib-cni-bin\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.113910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.113800 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-run-multus-certs\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.129620 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.129591 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-k6rqf" Jul 02 16:54:29.137747 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.137720 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-k6rqf" Jul 02 16:54:29.190505 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.190439 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-102.ec2.internal" event={"ID":"b66c94e3b0824384de48db496e306579","Type":"ContainerStarted","Data":"38a92cc6d9739accfedc2a8480148d7fa981377c22c85174673945f4eb4fbde3"} Jul 02 16:54:29.191831 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.191802 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal" event={"ID":"5353c433e8c4b5cfc4a4ccb34b7b3e4d","Type":"ContainerStarted","Data":"1a7506ded6c4703750617fa7f6edea789c4bece42ae4902afa0877871217e136"} Jul 02 16:54:29.198502 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.198479 2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 16:54:29.214015 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.213984 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-modprobe-d\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.214177 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214023 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-kubernetes\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.214177 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214046 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-etc-selinux\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.214177 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214066 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-var-lib-openvswitch\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.214177 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214098 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-cni-bin\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.214177 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214121 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa88be7d-b349-4071-8f78-2cdafc2fd875-cnibin\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.214177 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214142 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-registration-dir\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.214488 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214178 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa88be7d-b349-4071-8f78-2cdafc2fd875-system-cni-dir\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.214488 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214204 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5cf2\" (UniqueName: \"kubernetes.io/projected/aa88be7d-b349-4071-8f78-2cdafc2fd875-kube-api-access-v5cf2\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.214488 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214213 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-modprobe-d\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.214488 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214233 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k45j\" (UniqueName: \"kubernetes.io/projected/6c35723b-8e1e-4682-b963-70e3266a6a71-kube-api-access-8k45j\") pod \"network-metrics-daemon-bzrql\" (UID: \"6c35723b-8e1e-4682-b963-70e3266a6a71\") " pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:29.214488 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214260 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-multus-conf-dir\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.214488 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214286 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wst2s\" (UniqueName: \"kubernetes.io/projected/cce0691f-7213-4afa-b73e-d22536b1861a-kube-api-access-wst2s\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.214488 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214311 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-sysconfig\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.214488 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214334 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-systemd\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.214488 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214357 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-kubelet\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.214488 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214373 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-etc-selinux\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.214488 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214381 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-etc-kubernetes\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.214488 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214408 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-sys-fs\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.214488 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214423 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-var-lib-openvswitch\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.214488 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214440 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-run-systemd\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.214488 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214475 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-run-ovn\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214495 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa88be7d-b349-4071-8f78-2cdafc2fd875-cnibin\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214500 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-ovn-node-metrics-cert\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214524 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aa88be7d-b349-4071-8f78-2cdafc2fd875-os-release\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214551 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa88be7d-b349-4071-8f78-2cdafc2fd875-cni-binary-copy\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214554 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-registration-dir\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214596 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-multus-cni-dir\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214610 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa88be7d-b349-4071-8f78-2cdafc2fd875-system-cni-dir\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214619 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-lib-modules\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214644 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-log-socket\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214667 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbnlx\" (UniqueName: \"kubernetes.io/projected/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-kube-api-access-vbnlx\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214693 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aa88be7d-b349-4071-8f78-2cdafc2fd875-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214732 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz5xl\" (UniqueName: \"kubernetes.io/projected/28783fce-657d-4461-84ca-ae848eb09b15-kube-api-access-vz5xl\") pod \"node-resolver-6c5gj\" (UID: \"28783fce-657d-4461-84ca-ae848eb09b15\") " pod="openshift-dns/node-resolver-6c5gj" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214756 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-tuned\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214784 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214807 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-socket-dir\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214830 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-cni-netd\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.215180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214856 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214884 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa88be7d-b349-4071-8f78-2cdafc2fd875-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214917 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-multus-conf-dir\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214912 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aa88be7d-b349-4071-8f78-2cdafc2fd875-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214970 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dfb32fb2-9a93-4405-be31-8c26f66032e1-serviceca\") pod \"node-ca-clww4\" (UID: \"dfb32fb2-9a93-4405-be31-8c26f66032e1\") " pod="openshift-image-registry/node-ca-clww4" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214996 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-hostroot\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215022 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-sys\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215048 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-node-log\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215076 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98403e6f-9dcd-43ef-83f5-7b04bd39f933-host-slash\") pod \"iptables-alerter-cxfrf\" (UID: \"98403e6f-9dcd-43ef-83f5-7b04bd39f933\") " pod="openshift-network-operator/iptables-alerter-cxfrf" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215104 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs\") pod \"network-metrics-daemon-bzrql\" (UID: \"6c35723b-8e1e-4682-b963-70e3266a6a71\") " pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215137 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/28783fce-657d-4461-84ca-ae848eb09b15-hosts-file\") pod \"node-resolver-6c5gj\" (UID: \"28783fce-657d-4461-84ca-ae848eb09b15\") " pod="openshift-dns/node-resolver-6c5gj" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215180 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntfj8\" (UniqueName: \"kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8\") pod \"network-check-target-5xcjz\" (UID: \"3e206cf7-e305-45d7-be70-8b153b93bffa\") " pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215207 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dfb32fb2-9a93-4405-be31-8c26f66032e1-host\") pod \"node-ca-clww4\" (UID: \"dfb32fb2-9a93-4405-be31-8c26f66032e1\") " pod="openshift-image-registry/node-ca-clww4" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215236 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-run-k8s-cni-cncf-io\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215260 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-run\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215288 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-var-lib-kubelet\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215316 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-etc-openvswitch\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.215999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215347 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkfhj\" (UniqueName: \"kubernetes.io/projected/98403e6f-9dcd-43ef-83f5-7b04bd39f933-kube-api-access-wkfhj\") pod \"iptables-alerter-cxfrf\" (UID: \"98403e6f-9dcd-43ef-83f5-7b04bd39f933\") " pod="openshift-network-operator/iptables-alerter-cxfrf" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215374 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-system-cni-dir\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215401 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-multus-socket-dir-parent\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215437 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-var-lib-cni-bin\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215464 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-run-multus-certs\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215490 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-slash\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215518 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-run-ovn-kubernetes\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215532 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aa88be7d-b349-4071-8f78-2cdafc2fd875-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215545 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-run-netns\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.214285 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-cni-bin\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215575 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-device-dir\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215629 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9cc\" (UniqueName: \"kubernetes.io/projected/30310ad4-f069-4707-bef7-47f64a734db4-kube-api-access-9h9cc\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215651 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-device-dir\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215658 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-env-overrides\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215692 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-ovnkube-script-lib\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215731 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28783fce-657d-4461-84ca-ae848eb09b15-tmp-dir\") pod \"node-resolver-6c5gj\" (UID: \"28783fce-657d-4461-84ca-ae848eb09b15\") " pod="openshift-dns/node-resolver-6c5gj" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215758 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cce0691f-7213-4afa-b73e-d22536b1861a-cni-binary-copy\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.216744 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215783 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-sysctl-conf\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215809 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-systemd-units\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215834 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-host\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215859 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-run-netns\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215892 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3a271183-1ef5-49c2-a88f-7ea46b2b8864-kubelet-config\") pod \"global-pull-secret-syncer-76pzw\" (UID: \"3a271183-1ef5-49c2-a88f-7ea46b2b8864\") " pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215918 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-run-openvswitch\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215952 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-ovnkube-config\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.215977 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/53d27a00-961d-4ea1-9312-39cfc71e8bb5-agent-certs\") pod \"konnectivity-agent-wbgfs\" (UID: \"53d27a00-961d-4ea1-9312-39cfc71e8bb5\") " pod="kube-system/konnectivity-agent-wbgfs" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216000 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-cnibin\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216024 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-os-release\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216051 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-var-lib-kubelet\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216071 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dfb32fb2-9a93-4405-be31-8c26f66032e1-serviceca\") pod \"node-ca-clww4\" (UID: \"dfb32fb2-9a93-4405-be31-8c26f66032e1\") " pod="openshift-image-registry/node-ca-clww4" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216075 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3a271183-1ef5-49c2-a88f-7ea46b2b8864-dbus\") pod \"global-pull-secret-syncer-76pzw\" (UID: \"3a271183-1ef5-49c2-a88f-7ea46b2b8864\") " pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216110 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eca9108d-442e-458e-80cb-8a5f5e91885a-tmp\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216142 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/98403e6f-9dcd-43ef-83f5-7b04bd39f933-iptables-alerter-script\") pod \"iptables-alerter-cxfrf\" (UID: \"98403e6f-9dcd-43ef-83f5-7b04bd39f933\") " pod="openshift-network-operator/iptables-alerter-cxfrf" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216193 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/53d27a00-961d-4ea1-9312-39cfc71e8bb5-konnectivity-ca\") pod \"konnectivity-agent-wbgfs\" (UID: \"53d27a00-961d-4ea1-9312-39cfc71e8bb5\") " pod="kube-system/konnectivity-agent-wbgfs" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216224 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfdzd\" (UniqueName: \"kubernetes.io/projected/dfb32fb2-9a93-4405-be31-8c26f66032e1-kube-api-access-rfdzd\") pod \"node-ca-clww4\" (UID: \"dfb32fb2-9a93-4405-be31-8c26f66032e1\") " pod="openshift-image-registry/node-ca-clww4" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216253 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-var-lib-cni-multus\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.217486 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216290 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret\") pod \"global-pull-secret-syncer-76pzw\" (UID: \"3a271183-1ef5-49c2-a88f-7ea46b2b8864\") " pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216303 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3a271183-1ef5-49c2-a88f-7ea46b2b8864-dbus\") pod \"global-pull-secret-syncer-76pzw\" (UID: \"3a271183-1ef5-49c2-a88f-7ea46b2b8864\") " pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216318 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-sysctl-d\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216321 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-kubernetes\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216369 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-sysconfig\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216378 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-hostroot\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216411 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n78qs\" (UniqueName: \"kubernetes.io/projected/eca9108d-442e-458e-80cb-8a5f5e91885a-kube-api-access-n78qs\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216427 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-sysctl-d\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216432 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-sys\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216443 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cce0691f-7213-4afa-b73e-d22536b1861a-multus-daemon-config\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216474 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-system-cni-dir\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216491 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-multus-socket-dir-parent\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216500 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-node-log\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216539 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-systemd\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216548 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-var-lib-cni-bin\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216555 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-run-systemd\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216573 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-etc-kubernetes\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216605 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-run-ovn\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.218135 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216606 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-run-k8s-cni-cncf-io\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216636 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-run\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216643 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-slash\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216684 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/28783fce-657d-4461-84ca-ae848eb09b15-hosts-file\") pod \"node-resolver-6c5gj\" (UID: \"28783fce-657d-4461-84ca-ae848eb09b15\") " pod="openshift-dns/node-resolver-6c5gj" Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216693 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-kubelet\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216732 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-run-multus-certs\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216757 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dfb32fb2-9a93-4405-be31-8c26f66032e1-host\") pod \"node-ca-clww4\" (UID: \"dfb32fb2-9a93-4405-be31-8c26f66032e1\") " pod="openshift-image-registry/node-ca-clww4" Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216754 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-run-ovn-kubernetes\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216807 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-sys-fs\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.216846 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.216929 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret podName:3a271183-1ef5-49c2-a88f-7ea46b2b8864 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:29.716908152 +0000 UTC m=+3.029781790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret") pod "global-pull-secret-syncer-76pzw" (UID: "3a271183-1ef5-49c2-a88f-7ea46b2b8864") : object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.216956 2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217106 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-var-lib-kubelet\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217213 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98403e6f-9dcd-43ef-83f5-7b04bd39f933-host-slash\") pod \"iptables-alerter-cxfrf\" (UID: \"98403e6f-9dcd-43ef-83f5-7b04bd39f933\") " pod="openshift-network-operator/iptables-alerter-cxfrf" Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217235 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-etc-openvswitch\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217241 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3a271183-1ef5-49c2-a88f-7ea46b2b8864-kubelet-config\") pod \"global-pull-secret-syncer-76pzw\" (UID: \"3a271183-1ef5-49c2-a88f-7ea46b2b8864\") " pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217296 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-systemd-units\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217345 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-sysctl-conf\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.218894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217354 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-host\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217399 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-run-netns\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217420 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cce0691f-7213-4afa-b73e-d22536b1861a-multus-daemon-config\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217452 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-cnibin\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217479 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-multus-cni-dir\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217488 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28783fce-657d-4461-84ca-ae848eb09b15-tmp-dir\") pod \"node-resolver-6c5gj\" (UID: \"28783fce-657d-4461-84ca-ae848eb09b15\") " pod="openshift-dns/node-resolver-6c5gj" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217494 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-run-openvswitch\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217519 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-run-netns\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217567 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-var-lib-kubelet\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217608 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-os-release\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217683 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217723 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217824 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/30310ad4-f069-4707-bef7-47f64a734db4-socket-dir\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217837 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-ovnkube-script-lib\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217837 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cce0691f-7213-4afa-b73e-d22536b1861a-cni-binary-copy\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217863 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-host-cni-netd\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217916 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aa88be7d-b349-4071-8f78-2cdafc2fd875-os-release\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.217938 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eca9108d-442e-458e-80cb-8a5f5e91885a-lib-modules\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.219742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.218117 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa88be7d-b349-4071-8f78-2cdafc2fd875-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.220572 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.218183 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cce0691f-7213-4afa-b73e-d22536b1861a-host-var-lib-cni-multus\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.220572 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.218315 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/53d27a00-961d-4ea1-9312-39cfc71e8bb5-konnectivity-ca\") pod \"konnectivity-agent-wbgfs\" (UID: \"53d27a00-961d-4ea1-9312-39cfc71e8bb5\") " pod="kube-system/konnectivity-agent-wbgfs" Jul 02 16:54:29.220572 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.218322 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-env-overrides\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.220572 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.218397 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-log-socket\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.220572 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.218603 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa88be7d-b349-4071-8f78-2cdafc2fd875-cni-binary-copy\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.220572 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.218681 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-ovnkube-config\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.220572 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.218951 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aa88be7d-b349-4071-8f78-2cdafc2fd875-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.220572 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.219133 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/98403e6f-9dcd-43ef-83f5-7b04bd39f933-iptables-alerter-script\") pod \"iptables-alerter-cxfrf\" (UID: \"98403e6f-9dcd-43ef-83f5-7b04bd39f933\") " pod="openshift-network-operator/iptables-alerter-cxfrf" Jul 02 16:54:29.221453 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.221426 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/53d27a00-961d-4ea1-9312-39cfc71e8bb5-agent-certs\") pod \"konnectivity-agent-wbgfs\" (UID: \"53d27a00-961d-4ea1-9312-39cfc71e8bb5\") " pod="kube-system/konnectivity-agent-wbgfs" Jul 02 16:54:29.221711 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.221692 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eca9108d-442e-458e-80cb-8a5f5e91885a-etc-tuned\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.223303 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.223281 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eca9108d-442e-458e-80cb-8a5f5e91885a-tmp\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.223404 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.223347 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-ovn-node-metrics-cert\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.225420 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.225401 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5cf2\" (UniqueName: \"kubernetes.io/projected/aa88be7d-b349-4071-8f78-2cdafc2fd875-kube-api-access-v5cf2\") pod \"multus-additional-cni-plugins-fnsfd\" (UID: \"aa88be7d-b349-4071-8f78-2cdafc2fd875\") " pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.225620 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.225600 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 16:54:29.225620 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.225621 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 16:54:29.225775 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.225633 2556 projected.go:196] Error preparing data for projected volume kube-api-access-ntfj8 for pod openshift-network-diagnostics/network-check-target-5xcjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:29.225775 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.225764 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8 podName:3e206cf7-e305-45d7-be70-8b153b93bffa nodeName:}" failed. No retries permitted until 2026-07-02 16:54:29.725745348 +0000 UTC m=+3.038618995 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ntfj8" (UniqueName: "kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8") pod "network-check-target-5xcjz" (UID: "3e206cf7-e305-45d7-be70-8b153b93bffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:29.228265 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.228222 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkfhj\" (UniqueName: \"kubernetes.io/projected/98403e6f-9dcd-43ef-83f5-7b04bd39f933-kube-api-access-wkfhj\") pod \"iptables-alerter-cxfrf\" (UID: \"98403e6f-9dcd-43ef-83f5-7b04bd39f933\") " pod="openshift-network-operator/iptables-alerter-cxfrf" Jul 02 16:54:29.230339 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.230300 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wst2s\" (UniqueName: \"kubernetes.io/projected/cce0691f-7213-4afa-b73e-d22536b1861a-kube-api-access-wst2s\") pod \"multus-4mb2b\" (UID: \"cce0691f-7213-4afa-b73e-d22536b1861a\") " pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.230730 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.230643 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfdzd\" (UniqueName: \"kubernetes.io/projected/dfb32fb2-9a93-4405-be31-8c26f66032e1-kube-api-access-rfdzd\") pod \"node-ca-clww4\" (UID: \"dfb32fb2-9a93-4405-be31-8c26f66032e1\") " pod="openshift-image-registry/node-ca-clww4" Jul 02 16:54:29.231556 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.231054 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz5xl\" (UniqueName: \"kubernetes.io/projected/28783fce-657d-4461-84ca-ae848eb09b15-kube-api-access-vz5xl\") pod \"node-resolver-6c5gj\" (UID: \"28783fce-657d-4461-84ca-ae848eb09b15\") " pod="openshift-dns/node-resolver-6c5gj" Jul 02 16:54:29.231556 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.231227 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n78qs\" (UniqueName: \"kubernetes.io/projected/eca9108d-442e-458e-80cb-8a5f5e91885a-kube-api-access-n78qs\") pod \"tuned-ls7bj\" (UID: \"eca9108d-442e-458e-80cb-8a5f5e91885a\") " pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.231556 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.231266 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h9cc\" (UniqueName: \"kubernetes.io/projected/30310ad4-f069-4707-bef7-47f64a734db4-kube-api-access-9h9cc\") pod \"aws-ebs-csi-driver-node-dqp8t\" (UID: \"30310ad4-f069-4707-bef7-47f64a734db4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.233014 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.232953 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbnlx\" (UniqueName: \"kubernetes.io/projected/e0512e65-f357-4375-aa0d-6a30d8f2cbe4-kube-api-access-vbnlx\") pod \"ovnkube-node-qx6zp\" (UID: \"e0512e65-f357-4375-aa0d-6a30d8f2cbe4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.317741 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.317665 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k45j\" (UniqueName: \"kubernetes.io/projected/6c35723b-8e1e-4682-b963-70e3266a6a71-kube-api-access-8k45j\") pod \"network-metrics-daemon-bzrql\" (UID: \"6c35723b-8e1e-4682-b963-70e3266a6a71\") " pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:29.317741 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.317720 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs\") pod \"network-metrics-daemon-bzrql\" (UID: \"6c35723b-8e1e-4682-b963-70e3266a6a71\") " pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:29.317962 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.317828 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:29.317962 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.317948 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs podName:6c35723b-8e1e-4682-b963-70e3266a6a71 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:29.817926432 +0000 UTC m=+3.130800057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs") pod "network-metrics-daemon-bzrql" (UID: "6c35723b-8e1e-4682-b963-70e3266a6a71") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:29.326139 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.326113 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k45j\" (UniqueName: \"kubernetes.io/projected/6c35723b-8e1e-4682-b963-70e3266a6a71-kube-api-access-8k45j\") pod \"network-metrics-daemon-bzrql\" (UID: \"6c35723b-8e1e-4682-b963-70e3266a6a71\") " pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:29.339103 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.339074 2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 16:54:29.380224 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.380187 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" Jul 02 16:54:29.390196 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.390168 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-clww4" Jul 02 16:54:29.402141 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.402117 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:29.409001 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.408972 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4mb2b" Jul 02 16:54:29.416753 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.416735 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fnsfd" Jul 02 16:54:29.426301 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.426280 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cxfrf" Jul 02 16:54:29.433905 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.433879 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" Jul 02 16:54:29.442442 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.442424 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wbgfs" Jul 02 16:54:29.449027 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.449006 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6c5gj" Jul 02 16:54:29.720998 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.720921 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret\") pod \"global-pull-secret-syncer-76pzw\" (UID: \"3a271183-1ef5-49c2-a88f-7ea46b2b8864\") " pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:29.721143 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.721104 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:29.721219 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.721204 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret podName:3a271183-1ef5-49c2-a88f-7ea46b2b8864 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:30.721151785 +0000 UTC m=+4.034025408 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret") pod "global-pull-secret-syncer-76pzw" (UID: "3a271183-1ef5-49c2-a88f-7ea46b2b8864") : object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:29.822001 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.821962 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs\") pod \"network-metrics-daemon-bzrql\" (UID: \"6c35723b-8e1e-4682-b963-70e3266a6a71\") " pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:29.822001 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:29.821999 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntfj8\" (UniqueName: \"kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8\") pod \"network-check-target-5xcjz\" (UID: \"3e206cf7-e305-45d7-be70-8b153b93bffa\") " pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:29.822218 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.822125 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:29.822254 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.822226 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs podName:6c35723b-8e1e-4682-b963-70e3266a6a71 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:30.822201981 +0000 UTC m=+4.135075624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs") pod "network-metrics-daemon-bzrql" (UID: "6c35723b-8e1e-4682-b963-70e3266a6a71") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:29.822254 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.822228 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 16:54:29.822254 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.822246 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 16:54:29.822387 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.822258 2556 projected.go:196] Error preparing data for projected volume kube-api-access-ntfj8 for pod openshift-network-diagnostics/network-check-target-5xcjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:29.822387 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:29.822294 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8 podName:3e206cf7-e305-45d7-be70-8b153b93bffa nodeName:}" failed. No retries permitted until 2026-07-02 16:54:30.82228267 +0000 UTC m=+4.135156309 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ntfj8" (UniqueName: "kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8") pod "network-check-target-5xcjz" (UID: "3e206cf7-e305-45d7-be70-8b153b93bffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:30.141525 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:30.141485 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 16:49:29 +0000 UTC" deadline="2028-03-31 11:31:05.905157874 +0000 UTC" Jul 02 16:54:30.141525 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:30.141518 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15306h36m35.763645053s" Jul 02 16:54:30.183958 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:30.183920 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:30.184136 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:30.184075 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:30.380026 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:30.379996 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28783fce_657d_4461_84ca_ae848eb09b15.slice/crio-ca32f036bd0b6c1faa9ad6614b44d7be8c4861e6e1209d0ece555d1b3a86c6b7 WatchSource:0}: Error finding container ca32f036bd0b6c1faa9ad6614b44d7be8c4861e6e1209d0ece555d1b3a86c6b7: Status 404 returned error can't find the container with id ca32f036bd0b6c1faa9ad6614b44d7be8c4861e6e1209d0ece555d1b3a86c6b7 Jul 02 16:54:30.381011 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:30.380984 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53d27a00_961d_4ea1_9312_39cfc71e8bb5.slice/crio-383efc31ba43df3950b257d630b35aec43a7fd69f227894edd441927d7733d83 WatchSource:0}: Error finding container 383efc31ba43df3950b257d630b35aec43a7fd69f227894edd441927d7733d83: Status 404 returned error can't find the container with id 383efc31ba43df3950b257d630b35aec43a7fd69f227894edd441927d7733d83 Jul 02 16:54:30.385285 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:30.385022 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfb32fb2_9a93_4405_be31_8c26f66032e1.slice/crio-0484d3abf05088ed9f613a5241613b2ea4c7509d3c838b3d3f5e135eac49a3cd WatchSource:0}: Error finding container 0484d3abf05088ed9f613a5241613b2ea4c7509d3c838b3d3f5e135eac49a3cd: Status 404 returned error can't find the container with id 0484d3abf05088ed9f613a5241613b2ea4c7509d3c838b3d3f5e135eac49a3cd Jul 02 16:54:30.386438 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:30.386318 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98403e6f_9dcd_43ef_83f5_7b04bd39f933.slice/crio-f93e55df61a2f74094336a338ec13ff85e298379ff8735200bf3d40c4e9117ef WatchSource:0}: Error finding container f93e55df61a2f74094336a338ec13ff85e298379ff8735200bf3d40c4e9117ef: Status 404 returned error can't find the container with id f93e55df61a2f74094336a338ec13ff85e298379ff8735200bf3d40c4e9117ef Jul 02 16:54:30.386853 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:30.386787 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca9108d_442e_458e_80cb_8a5f5e91885a.slice/crio-de1e7e92cfac5bd75060dcb0a6a947b59985093d8d0406b3249bfe6759b01b55 WatchSource:0}: Error finding container de1e7e92cfac5bd75060dcb0a6a947b59985093d8d0406b3249bfe6759b01b55: Status 404 returned error can't find the container with id de1e7e92cfac5bd75060dcb0a6a947b59985093d8d0406b3249bfe6759b01b55 Jul 02 16:54:30.388595 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:30.388253 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30310ad4_f069_4707_bef7_47f64a734db4.slice/crio-15eb0da16341b074f1824168e12549528892c5a6c05b4494d1b9cb8f1574bafb WatchSource:0}: Error finding container 15eb0da16341b074f1824168e12549528892c5a6c05b4494d1b9cb8f1574bafb: Status 404 returned error can't find the container with id 15eb0da16341b074f1824168e12549528892c5a6c05b4494d1b9cb8f1574bafb Jul 02 16:54:30.390095 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:30.389710 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0512e65_f357_4375_aa0d_6a30d8f2cbe4.slice/crio-fad4bfeb7ebedbcfb5c29b39745994a45dc5887ef106656fd6c93aba09656a81 WatchSource:0}: Error finding container fad4bfeb7ebedbcfb5c29b39745994a45dc5887ef106656fd6c93aba09656a81: Status 404 returned error can't find the container with id fad4bfeb7ebedbcfb5c29b39745994a45dc5887ef106656fd6c93aba09656a81 Jul 02 16:54:30.391105 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:30.390526 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcce0691f_7213_4afa_b73e_d22536b1861a.slice/crio-7985710bde56cb36f891a2b9ef0f7132d62e54a8a5a1b091ce46353afb8ea98d WatchSource:0}: Error finding container 7985710bde56cb36f891a2b9ef0f7132d62e54a8a5a1b091ce46353afb8ea98d: Status 404 returned error can't find the container with id 7985710bde56cb36f891a2b9ef0f7132d62e54a8a5a1b091ce46353afb8ea98d Jul 02 16:54:30.391918 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:54:30.391818 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa88be7d_b349_4071_8f78_2cdafc2fd875.slice/crio-8d37bbd6bf351e5caa191372a6acbed2152243ee762baa3fdeee9127e8792650 WatchSource:0}: Error finding container 8d37bbd6bf351e5caa191372a6acbed2152243ee762baa3fdeee9127e8792650: Status 404 returned error can't find the container with id 8d37bbd6bf351e5caa191372a6acbed2152243ee762baa3fdeee9127e8792650 Jul 02 16:54:30.728023 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:30.727991 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret\") pod \"global-pull-secret-syncer-76pzw\" (UID: \"3a271183-1ef5-49c2-a88f-7ea46b2b8864\") " pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:30.728213 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:30.728105 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:30.728213 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:30.728178 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret podName:3a271183-1ef5-49c2-a88f-7ea46b2b8864 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:32.728147585 +0000 UTC m=+6.041021209 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret") pod "global-pull-secret-syncer-76pzw" (UID: "3a271183-1ef5-49c2-a88f-7ea46b2b8864") : object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:30.828684 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:30.828645 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs\") pod \"network-metrics-daemon-bzrql\" (UID: \"6c35723b-8e1e-4682-b963-70e3266a6a71\") " pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:30.828863 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:30.828698 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntfj8\" (UniqueName: \"kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8\") pod \"network-check-target-5xcjz\" (UID: \"3e206cf7-e305-45d7-be70-8b153b93bffa\") " pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:30.828927 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:30.828882 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 16:54:30.828927 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:30.828903 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 16:54:30.828927 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:30.828915 2556 projected.go:196] Error preparing data for projected volume kube-api-access-ntfj8 for pod openshift-network-diagnostics/network-check-target-5xcjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:30.829085 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:30.828975 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8 podName:3e206cf7-e305-45d7-be70-8b153b93bffa nodeName:}" failed. No retries permitted until 2026-07-02 16:54:32.828956468 +0000 UTC m=+6.141830115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ntfj8" (UniqueName: "kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8") pod "network-check-target-5xcjz" (UID: "3e206cf7-e305-45d7-be70-8b153b93bffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:30.829409 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:30.829389 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:30.829518 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:30.829447 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs podName:6c35723b-8e1e-4682-b963-70e3266a6a71 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:32.82942995 +0000 UTC m=+6.142303597 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs") pod "network-metrics-daemon-bzrql" (UID: "6c35723b-8e1e-4682-b963-70e3266a6a71") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:31.142354 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.142285 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 16:49:29 +0000 UTC" deadline="2027-11-28 20:46:00.031248935 +0000 UTC" Jul 02 16:54:31.142354 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.142351 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12339h51m28.888902182s" Jul 02 16:54:31.184778 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.184744 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:31.184959 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:31.184885 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:54:31.184959 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.184934 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:31.185087 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:31.185070 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:54:31.207423 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.207383 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-102.ec2.internal" event={"ID":"b66c94e3b0824384de48db496e306579","Type":"ContainerStarted","Data":"41d6ff6e886dc39f8a099d01d37acb9b5d46526a732110047602124d18a06a86"} Jul 02 16:54:31.222523 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.221733 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-102.ec2.internal" podStartSLOduration=3.221711602 podStartE2EDuration="3.221711602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:31.221463147 +0000 UTC m=+4.534336794" watchObservedRunningTime="2026-07-02 16:54:31.221711602 +0000 UTC m=+4.534585250" Jul 02 16:54:31.223915 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.223878 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fnsfd" event={"ID":"aa88be7d-b349-4071-8f78-2cdafc2fd875","Type":"ContainerStarted","Data":"8d37bbd6bf351e5caa191372a6acbed2152243ee762baa3fdeee9127e8792650"} Jul 02 16:54:31.233826 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.233791 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" event={"ID":"e0512e65-f357-4375-aa0d-6a30d8f2cbe4","Type":"ContainerStarted","Data":"fad4bfeb7ebedbcfb5c29b39745994a45dc5887ef106656fd6c93aba09656a81"} Jul 02 16:54:31.241014 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.240932 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4mb2b" event={"ID":"cce0691f-7213-4afa-b73e-d22536b1861a","Type":"ContainerStarted","Data":"7985710bde56cb36f891a2b9ef0f7132d62e54a8a5a1b091ce46353afb8ea98d"} Jul 02 16:54:31.243871 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.243797 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" event={"ID":"30310ad4-f069-4707-bef7-47f64a734db4","Type":"ContainerStarted","Data":"15eb0da16341b074f1824168e12549528892c5a6c05b4494d1b9cb8f1574bafb"} Jul 02 16:54:31.245936 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.245870 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" event={"ID":"eca9108d-442e-458e-80cb-8a5f5e91885a","Type":"ContainerStarted","Data":"de1e7e92cfac5bd75060dcb0a6a947b59985093d8d0406b3249bfe6759b01b55"} Jul 02 16:54:31.250474 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.250405 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wbgfs" event={"ID":"53d27a00-961d-4ea1-9312-39cfc71e8bb5","Type":"ContainerStarted","Data":"383efc31ba43df3950b257d630b35aec43a7fd69f227894edd441927d7733d83"} Jul 02 16:54:31.256053 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.255238 2556 generic.go:356] "Generic (PLEG): container finished" podID="5353c433e8c4b5cfc4a4ccb34b7b3e4d" containerID="0931a6b6b882d7e3f53f51ac755d5cf420ab25ebfe9078b453fe9a56668cc1f8" exitCode=0 Jul 02 16:54:31.256053 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.255320 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal" event={"ID":"5353c433e8c4b5cfc4a4ccb34b7b3e4d","Type":"ContainerDied","Data":"0931a6b6b882d7e3f53f51ac755d5cf420ab25ebfe9078b453fe9a56668cc1f8"} Jul 02 16:54:31.263842 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.263772 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cxfrf" event={"ID":"98403e6f-9dcd-43ef-83f5-7b04bd39f933","Type":"ContainerStarted","Data":"f93e55df61a2f74094336a338ec13ff85e298379ff8735200bf3d40c4e9117ef"} Jul 02 16:54:31.274943 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.274881 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-clww4" event={"ID":"dfb32fb2-9a93-4405-be31-8c26f66032e1","Type":"ContainerStarted","Data":"0484d3abf05088ed9f613a5241613b2ea4c7509d3c838b3d3f5e135eac49a3cd"} Jul 02 16:54:31.284720 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:31.284669 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6c5gj" event={"ID":"28783fce-657d-4461-84ca-ae848eb09b15","Type":"ContainerStarted","Data":"ca32f036bd0b6c1faa9ad6614b44d7be8c4861e6e1209d0ece555d1b3a86c6b7"} Jul 02 16:54:32.183871 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:32.183839 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:32.184655 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:32.183970 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:32.306787 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:32.306731 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal" event={"ID":"5353c433e8c4b5cfc4a4ccb34b7b3e4d","Type":"ContainerStarted","Data":"4dfa9425b5aac1b835da883579977c96f50c833f736891bd0dcc8655c47abee1"} Jul 02 16:54:32.322116 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:32.321555 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-102.ec2.internal" podStartSLOduration=4.32153448 podStartE2EDuration="4.32153448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:32.321289582 +0000 UTC m=+5.634163229" watchObservedRunningTime="2026-07-02 16:54:32.32153448 +0000 UTC m=+5.634408127" Jul 02 16:54:32.744212 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:32.743570 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret\") pod \"global-pull-secret-syncer-76pzw\" (UID: \"3a271183-1ef5-49c2-a88f-7ea46b2b8864\") " pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:32.744212 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:32.743780 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:32.744212 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:32.743841 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret podName:3a271183-1ef5-49c2-a88f-7ea46b2b8864 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:36.743823549 +0000 UTC m=+10.056697173 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret") pod "global-pull-secret-syncer-76pzw" (UID: "3a271183-1ef5-49c2-a88f-7ea46b2b8864") : object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:32.845529 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:32.844525 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs\") pod \"network-metrics-daemon-bzrql\" (UID: \"6c35723b-8e1e-4682-b963-70e3266a6a71\") " pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:32.845529 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:32.844716 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntfj8\" (UniqueName: \"kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8\") pod \"network-check-target-5xcjz\" (UID: \"3e206cf7-e305-45d7-be70-8b153b93bffa\") " pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:32.845529 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:32.844909 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 16:54:32.845529 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:32.844932 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 16:54:32.845529 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:32.844946 2556 projected.go:196] Error preparing data for projected volume kube-api-access-ntfj8 for pod openshift-network-diagnostics/network-check-target-5xcjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:32.845529 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:32.845005 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8 podName:3e206cf7-e305-45d7-be70-8b153b93bffa nodeName:}" failed. No retries permitted until 2026-07-02 16:54:36.844986896 +0000 UTC m=+10.157860541 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ntfj8" (UniqueName: "kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8") pod "network-check-target-5xcjz" (UID: "3e206cf7-e305-45d7-be70-8b153b93bffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:32.845529 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:32.845442 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:32.845529 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:32.845494 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs podName:6c35723b-8e1e-4682-b963-70e3266a6a71 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:36.84547716 +0000 UTC m=+10.158350816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs") pod "network-metrics-daemon-bzrql" (UID: "6c35723b-8e1e-4682-b963-70e3266a6a71") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:33.183747 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:33.183716 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:33.183945 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:33.183834 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:54:33.184409 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:33.184238 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:33.184409 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:33.184348 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:54:34.184110 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:34.184074 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:34.184631 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:34.184238 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:35.187487 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:35.187454 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:35.187928 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:35.187512 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:35.187928 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:35.187594 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:54:35.187928 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:35.187719 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:54:36.184909 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:36.184397 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:36.184909 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:36.184548 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:36.778699 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:36.778658 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret\") pod \"global-pull-secret-syncer-76pzw\" (UID: \"3a271183-1ef5-49c2-a88f-7ea46b2b8864\") " pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:36.779193 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:36.778857 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:36.779193 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:36.778938 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret podName:3a271183-1ef5-49c2-a88f-7ea46b2b8864 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:44.778917183 +0000 UTC m=+18.091790822 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret") pod "global-pull-secret-syncer-76pzw" (UID: "3a271183-1ef5-49c2-a88f-7ea46b2b8864") : object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:36.880129 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:36.879912 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs\") pod \"network-metrics-daemon-bzrql\" (UID: \"6c35723b-8e1e-4682-b963-70e3266a6a71\") " pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:36.880129 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:36.879976 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntfj8\" (UniqueName: \"kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8\") pod \"network-check-target-5xcjz\" (UID: \"3e206cf7-e305-45d7-be70-8b153b93bffa\") " pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:36.881537 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:36.881445 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 16:54:36.881537 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:36.881474 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 16:54:36.881537 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:36.881491 2556 projected.go:196] Error preparing data for projected volume kube-api-access-ntfj8 for pod openshift-network-diagnostics/network-check-target-5xcjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:36.881537 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:36.881491 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:36.881821 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:36.881556 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8 podName:3e206cf7-e305-45d7-be70-8b153b93bffa nodeName:}" failed. No retries permitted until 2026-07-02 16:54:44.881527445 +0000 UTC m=+18.194401070 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ntfj8" (UniqueName: "kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8") pod "network-check-target-5xcjz" (UID: "3e206cf7-e305-45d7-be70-8b153b93bffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:36.881821 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:36.881577 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs podName:6c35723b-8e1e-4682-b963-70e3266a6a71 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:44.881567212 +0000 UTC m=+18.194440837 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs") pod "network-metrics-daemon-bzrql" (UID: "6c35723b-8e1e-4682-b963-70e3266a6a71") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:37.185143 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:37.184562 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:37.185143 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:37.184699 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:54:37.185143 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:37.184718 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:37.185143 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:37.184854 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:54:38.183826 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:38.183790 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:38.184289 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:38.183921 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:39.184287 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:39.184260 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:39.184287 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:39.184273 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:39.184799 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:39.184403 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:54:39.184799 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:39.184539 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:54:40.184170 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:40.184130 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:40.184360 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:40.184283 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:41.183525 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:41.183487 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:41.183668 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:41.183605 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:54:41.183668 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:41.183659 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:41.183755 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:41.183737 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:54:42.184303 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:42.184271 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:42.184723 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:42.184390 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:43.184434 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:43.184401 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:43.184852 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:43.184401 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:43.184852 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:43.184511 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:54:43.184852 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:43.184612 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:54:44.184071 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:44.184034 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:44.184326 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:44.184181 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:44.842570 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:44.842532 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret\") pod \"global-pull-secret-syncer-76pzw\" (UID: \"3a271183-1ef5-49c2-a88f-7ea46b2b8864\") " pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:44.843035 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:44.842704 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:44.843035 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:44.842789 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret podName:3a271183-1ef5-49c2-a88f-7ea46b2b8864 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:00.842768188 +0000 UTC m=+34.155641817 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret") pod "global-pull-secret-syncer-76pzw" (UID: "3a271183-1ef5-49c2-a88f-7ea46b2b8864") : object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:44.943830 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:44.943791 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs\") pod \"network-metrics-daemon-bzrql\" (UID: \"6c35723b-8e1e-4682-b963-70e3266a6a71\") " pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:44.943830 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:44.943836 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntfj8\" (UniqueName: \"kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8\") pod \"network-check-target-5xcjz\" (UID: \"3e206cf7-e305-45d7-be70-8b153b93bffa\") " pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:44.944052 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:44.943970 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:44.944052 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:44.944012 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 16:54:44.944052 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:44.944036 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 16:54:44.944052 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:44.944050 2556 projected.go:196] Error preparing data for projected volume kube-api-access-ntfj8 for pod openshift-network-diagnostics/network-check-target-5xcjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:44.944245 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:44.944052 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs podName:6c35723b-8e1e-4682-b963-70e3266a6a71 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:00.944030003 +0000 UTC m=+34.256903640 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs") pod "network-metrics-daemon-bzrql" (UID: "6c35723b-8e1e-4682-b963-70e3266a6a71") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:44.944245 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:44.944137 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8 podName:3e206cf7-e305-45d7-be70-8b153b93bffa nodeName:}" failed. No retries permitted until 2026-07-02 16:55:00.944111417 +0000 UTC m=+34.256985048 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ntfj8" (UniqueName: "kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8") pod "network-check-target-5xcjz" (UID: "3e206cf7-e305-45d7-be70-8b153b93bffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:45.184024 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:45.183933 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:45.184024 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:45.183959 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:45.184263 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:45.184074 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:54:45.184361 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:45.184320 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:54:46.183982 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:46.183944 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:46.184411 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:46.184077 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:47.184876 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:47.184845 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:47.185342 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:47.184985 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:54:47.185342 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:47.185025 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:47.185342 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:47.185132 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:54:48.184203 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:48.184152 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:48.184401 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:48.184284 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:49.184844 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:49.184333 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:49.184844 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:49.184490 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:54:49.184844 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:49.184332 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:49.184844 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:49.184642 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:54:49.344823 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:49.344594 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-clww4" event={"ID":"dfb32fb2-9a93-4405-be31-8c26f66032e1","Type":"ContainerStarted","Data":"1e5a8a7df7144df34efcf20b15b2e0438c1653ed15046f106949df9b9ff0cef2"} Jul 02 16:54:49.347597 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:49.347450 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fnsfd" event={"ID":"aa88be7d-b349-4071-8f78-2cdafc2fd875","Type":"ContainerStarted","Data":"7c0f0b0818deced0da0703ee1fb5f6c2ebf3433ea3ae2c456cab2f607119b098"} Jul 02 16:54:49.349963 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:49.349842 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" event={"ID":"e0512e65-f357-4375-aa0d-6a30d8f2cbe4","Type":"ContainerStarted","Data":"dffbdb01519fec62249139e96061d5d793b8e75acab0c5efd4e575c662093eea"} Jul 02 16:54:49.353104 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:49.352499 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4mb2b" event={"ID":"cce0691f-7213-4afa-b73e-d22536b1861a","Type":"ContainerStarted","Data":"38626d50d6eb07dc773446acc03c9c2db561a1da864af03916976792572de653"} Jul 02 16:54:49.357137 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:49.357106 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" event={"ID":"eca9108d-442e-458e-80cb-8a5f5e91885a","Type":"ContainerStarted","Data":"b1289073b92429a0836a8fff4e62292a6d622ab35e5cd22126790ad6037e3202"} Jul 02 16:54:49.359047 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:49.358996 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wbgfs" event={"ID":"53d27a00-961d-4ea1-9312-39cfc71e8bb5","Type":"ContainerStarted","Data":"56d46c942528ad2beaddbfb0fd0676e7da109b797c6d41025c53002c5458f979"} Jul 02 16:54:49.361623 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:49.361519 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-clww4" podStartSLOduration=3.628344458 podStartE2EDuration="22.361499856s" totalImagesPullingTime="18.733155398s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:49.361029903 +0000 UTC m=+22.673903562" watchObservedRunningTime="2026-07-02 16:54:49.361499856 +0000 UTC m=+22.674373502" Jul 02 16:54:49.377343 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:49.377151 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4mb2b" podStartSLOduration=3.593609631 podStartE2EDuration="22.377136219s" totalImagesPullingTime="18.783526588s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:49.377075122 +0000 UTC m=+22.689948759" watchObservedRunningTime="2026-07-02 16:54:49.377136219 +0000 UTC m=+22.690009865" Jul 02 16:54:49.394552 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:49.394458 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ls7bj" podStartSLOduration=3.642626847 podStartE2EDuration="22.394427763s" totalImagesPullingTime="18.751800916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:49.394342989 +0000 UTC m=+22.707216637" watchObservedRunningTime="2026-07-02 16:54:49.394427763 +0000 UTC m=+22.707301411" Jul 02 16:54:49.409213 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:49.409026 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wbgfs" podStartSLOduration=3.751567191 podStartE2EDuration="22.409012072s" totalImagesPullingTime="18.657444881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:49.408638987 +0000 UTC m=+22.721512634" watchObservedRunningTime="2026-07-02 16:54:49.409012072 +0000 UTC m=+22.721885717" Jul 02 16:54:49.786665 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:49.786487 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wbgfs" Jul 02 16:54:50.183648 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:50.183611 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:50.183825 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:50.183732 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:50.362395 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:50.362365 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6c5gj" event={"ID":"28783fce-657d-4461-84ca-ae848eb09b15","Type":"ContainerStarted","Data":"82fdf1013a7e37b7421ccc685667960a7d057032cddc67e8877d54bf72ee210d"} Jul 02 16:54:50.363906 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:50.363888 2556 generic.go:356] "Generic (PLEG): container finished" podID="aa88be7d-b349-4071-8f78-2cdafc2fd875" containerID="7c0f0b0818deced0da0703ee1fb5f6c2ebf3433ea3ae2c456cab2f607119b098" exitCode=0 Jul 02 16:54:50.363969 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:50.363960 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fnsfd" event={"ID":"aa88be7d-b349-4071-8f78-2cdafc2fd875","Type":"ContainerDied","Data":"7c0f0b0818deced0da0703ee1fb5f6c2ebf3433ea3ae2c456cab2f607119b098"} Jul 02 16:54:50.367025 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:50.366998 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" event={"ID":"e0512e65-f357-4375-aa0d-6a30d8f2cbe4","Type":"ContainerStarted","Data":"66915f76c82172f417d2ee4b6da4db6d2db53ddb8644eb0d016be5bf520df426"} Jul 02 16:54:50.367134 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:50.367061 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" event={"ID":"e0512e65-f357-4375-aa0d-6a30d8f2cbe4","Type":"ContainerStarted","Data":"6341aead6ec1bae337d9b64377988bc98fb7d4e35ced181f45558d77b82cf4bb"} Jul 02 16:54:50.367134 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:50.367092 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" event={"ID":"e0512e65-f357-4375-aa0d-6a30d8f2cbe4","Type":"ContainerStarted","Data":"7bf99c26f918232d25ffd620f890c86f422a9884e201112b824226abdfcda681"} Jul 02 16:54:50.367134 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:50.367131 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" event={"ID":"e0512e65-f357-4375-aa0d-6a30d8f2cbe4","Type":"ContainerStarted","Data":"de6e883082aad0359ffebd7ecd4816f291e542c3ccdb20de0c3a3820c62f7064"} Jul 02 16:54:50.367305 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:50.367176 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" event={"ID":"e0512e65-f357-4375-aa0d-6a30d8f2cbe4","Type":"ContainerStarted","Data":"d953e5d5a9c7a073837bf95121ad28736814cdf7dfb8ea26bc845c577a67f016"} Jul 02 16:54:50.368439 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:50.368416 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" event={"ID":"30310ad4-f069-4707-bef7-47f64a734db4","Type":"ContainerStarted","Data":"393f3ae11547f69c92caf150a53fe9f6a890caf922d23d85998b4de4ad3bd57f"} Jul 02 16:54:50.377358 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:50.377318 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6c5gj" podStartSLOduration=4.636288648 podStartE2EDuration="23.37730399s" totalImagesPullingTime="18.741015342s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:50.376862186 +0000 UTC m=+23.689735833" watchObservedRunningTime="2026-07-02 16:54:50.37730399 +0000 UTC m=+23.690177635" Jul 02 16:54:50.611502 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:50.611480 2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 16:54:51.184669 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:51.184141 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:51.184669 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:51.184152 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:51.184669 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:51.184296 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:54:51.184669 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:51.184394 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:54:51.186482 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:51.186382 2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T16:54:50.611498476Z","UUID":"f42a3c83-ee3e-412e-9c15-9b0a30c69d93","Handler":null,"Name":"","Endpoint":""} Jul 02 16:54:51.188275 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:51.188241 2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 16:54:51.188275 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:51.188277 2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 16:54:51.373172 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:51.373116 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" event={"ID":"30310ad4-f069-4707-bef7-47f64a734db4","Type":"ContainerStarted","Data":"3df99068bbb856c847bc2c2acf14698dc77b7aa4c9be741bf0dbf70195954504"} Jul 02 16:54:51.375138 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:51.375109 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cxfrf" event={"ID":"98403e6f-9dcd-43ef-83f5-7b04bd39f933","Type":"ContainerStarted","Data":"6f46779aee67cc19f5b444f1aa09f815d402cfcbf6390f8461137b077c9d91be"} Jul 02 16:54:51.388761 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:51.388707 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cxfrf" podStartSLOduration=5.655793565 podStartE2EDuration="24.388694783s" totalImagesPullingTime="18.732901218s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:51.388220995 +0000 UTC m=+24.701094644" watchObservedRunningTime="2026-07-02 16:54:51.388694783 +0000 UTC m=+24.701568429" Jul 02 16:54:52.184090 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:52.184056 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:52.184368 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:52.184186 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:52.381844 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:52.381807 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" event={"ID":"e0512e65-f357-4375-aa0d-6a30d8f2cbe4","Type":"ContainerStarted","Data":"45623287cd80138d388d391e8be59e4a125caf7e9f5525c4c3b567375ded7fe4"} Jul 02 16:54:52.384149 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:52.384114 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" event={"ID":"30310ad4-f069-4707-bef7-47f64a734db4","Type":"ContainerStarted","Data":"cb5d7dac316352d36ab6646be3764257132a02ce1da870c78459a75cb02c2d0f"} Jul 02 16:54:52.400209 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:52.400152 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dqp8t" podStartSLOduration=4.460607532 podStartE2EDuration="25.400138119s" totalImagesPullingTime="20.939530587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:52.399806944 +0000 UTC m=+25.712680591" watchObservedRunningTime="2026-07-02 16:54:52.400138119 +0000 UTC m=+25.713011765" Jul 02 16:54:53.183480 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:53.183441 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:53.183480 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:53.183476 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:53.183737 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:53.183593 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:54:53.183799 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:53.183731 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:54:54.183600 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:54.183551 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:54.184075 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:54.183698 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:54.226697 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:54.226661 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wbgfs" Jul 02 16:54:54.227366 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:54.227341 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wbgfs" Jul 02 16:54:54.392226 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:54.392078 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wbgfs" Jul 02 16:54:55.183646 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:55.183616 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:55.184415 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:55.183616 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:55.184415 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:55.183722 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:54:55.184415 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:55.183812 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:54:55.394764 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:55.394731 2556 generic.go:356] "Generic (PLEG): container finished" podID="aa88be7d-b349-4071-8f78-2cdafc2fd875" containerID="ef016fd3c331743e510360d0f615c632021c0e552dab6092df40578694ff65b8" exitCode=0 Jul 02 16:54:55.394996 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:55.394814 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fnsfd" event={"ID":"aa88be7d-b349-4071-8f78-2cdafc2fd875","Type":"ContainerDied","Data":"ef016fd3c331743e510360d0f615c632021c0e552dab6092df40578694ff65b8"} Jul 02 16:54:55.398618 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:55.398593 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" event={"ID":"e0512e65-f357-4375-aa0d-6a30d8f2cbe4","Type":"ContainerStarted","Data":"ef1626f84fdac13c2bbd3a06342c30af478c7f1f6c4de221d4108c8dfc56bee0"} Jul 02 16:54:55.398785 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:55.398767 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:55.442207 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:55.442133 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" podStartSLOduration=9.639637369 podStartE2EDuration="28.442114091s" totalImagesPullingTime="18.802476722s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:55.441664881 +0000 UTC m=+28.754538526" watchObservedRunningTime="2026-07-02 16:54:55.442114091 +0000 UTC m=+28.754987737" Jul 02 16:54:56.183655 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:56.183466 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:56.184133 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:56.183734 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:56.401774 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:56.401733 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:56.401958 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:56.401833 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:56.419395 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:56.419368 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:56.419558 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:56.419487 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:54:56.661515 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:56.661477 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5xcjz"] Jul 02 16:54:56.661691 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:56.661660 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:56.661799 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:56.661775 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:54:56.662384 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:56.662320 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bzrql"] Jul 02 16:54:56.662542 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:56.662491 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:56.662637 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:56.662613 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:54:56.663057 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:56.663034 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-76pzw"] Jul 02 16:54:56.663180 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:56.663138 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:56.663317 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:56.663298 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:57.405667 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:57.405633 2556 generic.go:356] "Generic (PLEG): container finished" podID="aa88be7d-b349-4071-8f78-2cdafc2fd875" containerID="ce1bd0803c69c192eef1b5fb70eac62a35cb2dade7b29118d029157dba754e20" exitCode=0 Jul 02 16:54:57.406082 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:57.405721 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fnsfd" event={"ID":"aa88be7d-b349-4071-8f78-2cdafc2fd875","Type":"ContainerDied","Data":"ce1bd0803c69c192eef1b5fb70eac62a35cb2dade7b29118d029157dba754e20"} Jul 02 16:54:58.183463 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:58.183438 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:54:58.183581 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:58.183545 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:54:58.183581 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:58.183557 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:54:58.183581 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:58.183573 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:54:58.183736 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:58.183651 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:54:58.183736 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:54:58.183716 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:54:58.411040 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:58.411005 2556 generic.go:356] "Generic (PLEG): container finished" podID="aa88be7d-b349-4071-8f78-2cdafc2fd875" containerID="295df80d56aeb6ed5c246e9af224e727c51b0622c038155af6ad96825fc62507" exitCode=0 Jul 02 16:54:58.411427 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:54:58.411231 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fnsfd" event={"ID":"aa88be7d-b349-4071-8f78-2cdafc2fd875","Type":"ContainerDied","Data":"295df80d56aeb6ed5c246e9af224e727c51b0622c038155af6ad96825fc62507"} Jul 02 16:55:00.183818 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:00.183782 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:55:00.183818 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:00.183800 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:55:00.184363 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:00.183782 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:55:00.184363 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:00.183920 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:55:00.184363 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:00.184014 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-76pzw" podUID="3a271183-1ef5-49c2-a88f-7ea46b2b8864" Jul 02 16:55:00.184363 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:00.184085 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5xcjz" podUID="3e206cf7-e305-45d7-be70-8b153b93bffa" Jul 02 16:55:00.856715 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:00.856478 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret\") pod \"global-pull-secret-syncer-76pzw\" (UID: \"3a271183-1ef5-49c2-a88f-7ea46b2b8864\") " pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:55:00.856955 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:00.856635 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 16:55:00.856955 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:00.856868 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret podName:3a271183-1ef5-49c2-a88f-7ea46b2b8864 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:32.856843797 +0000 UTC m=+66.169717423 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret") pod "global-pull-secret-syncer-76pzw" (UID: "3a271183-1ef5-49c2-a88f-7ea46b2b8864") : object "kube-system"/"original-pull-secret" not registered Jul 02 16:55:00.957732 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:00.957691 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs\") pod \"network-metrics-daemon-bzrql\" (UID: \"6c35723b-8e1e-4682-b963-70e3266a6a71\") " pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:55:00.957732 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:00.957736 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntfj8\" (UniqueName: \"kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8\") pod \"network-check-target-5xcjz\" (UID: \"3e206cf7-e305-45d7-be70-8b153b93bffa\") " pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:55:00.957973 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:00.957847 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:55:00.957973 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:00.957877 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 16:55:00.957973 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:00.957892 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 16:55:00.957973 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:00.957905 2556 projected.go:196] Error preparing data for projected volume kube-api-access-ntfj8 for pod openshift-network-diagnostics/network-check-target-5xcjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:55:00.957973 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:00.957921 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs podName:6c35723b-8e1e-4682-b963-70e3266a6a71 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:32.957900316 +0000 UTC m=+66.270773945 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs") pod "network-metrics-daemon-bzrql" (UID: "6c35723b-8e1e-4682-b963-70e3266a6a71") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:55:00.957973 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:00.957952 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8 podName:3e206cf7-e305-45d7-be70-8b153b93bffa nodeName:}" failed. No retries permitted until 2026-07-02 16:55:32.957940524 +0000 UTC m=+66.270814173 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ntfj8" (UniqueName: "kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8") pod "network-check-target-5xcjz" (UID: "3e206cf7-e305-45d7-be70-8b153b93bffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:55:01.969022 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:01.968934 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-102.ec2.internal" event="NodeReady" Jul 02 16:55:01.969511 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:01.969094 2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 16:55:02.008891 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.008856 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-76lqz"] Jul 02 16:55:02.033823 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.033790 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9qqxh"] Jul 02 16:55:02.034010 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.033941 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:02.036462 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.036436 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 16:55:02.036462 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.036458 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 16:55:02.036707 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.036498 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mmn65\"" Jul 02 16:55:02.056983 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.056950 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-76lqz"] Jul 02 16:55:02.057152 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.057011 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9qqxh"] Jul 02 16:55:02.057152 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.057092 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9qqxh" Jul 02 16:55:02.059715 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.059680 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 16:55:02.059715 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.059700 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-947jv\"" Jul 02 16:55:02.059922 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.059680 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 16:55:02.059922 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.059745 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 16:55:02.166422 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.166387 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cec11ca-b0a2-4bd4-ae8e-bb2e6ac12df0-cert\") pod \"ingress-canary-9qqxh\" (UID: \"6cec11ca-b0a2-4bd4-ae8e-bb2e6ac12df0\") " pod="openshift-ingress-canary/ingress-canary-9qqxh" Jul 02 16:55:02.166629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.166450 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4l62\" (UniqueName: \"kubernetes.io/projected/eca22f13-1929-48b6-90fd-5d82f58f6017-kube-api-access-h4l62\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:02.166629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.166524 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:02.166629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.166548 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eca22f13-1929-48b6-90fd-5d82f58f6017-tmp-dir\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:02.166629 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.166611 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6bjp\" (UniqueName: \"kubernetes.io/projected/6cec11ca-b0a2-4bd4-ae8e-bb2e6ac12df0-kube-api-access-n6bjp\") pod \"ingress-canary-9qqxh\" (UID: \"6cec11ca-b0a2-4bd4-ae8e-bb2e6ac12df0\") " pod="openshift-ingress-canary/ingress-canary-9qqxh" Jul 02 16:55:02.166824 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.166637 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eca22f13-1929-48b6-90fd-5d82f58f6017-config-volume\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:02.183434 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.183402 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:55:02.183568 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.183402 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:55:02.183636 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.183416 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:55:02.186137 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.186113 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 16:55:02.186272 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.186152 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 16:55:02.186272 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.186174 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 16:55:02.186272 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.186197 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 16:55:02.186272 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.186124 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6k5zx\"" Jul 02 16:55:02.186447 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.186327 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-442wp\"" Jul 02 16:55:02.267460 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.267428 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4l62\" (UniqueName: \"kubernetes.io/projected/eca22f13-1929-48b6-90fd-5d82f58f6017-kube-api-access-h4l62\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:02.267647 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.267516 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:02.267647 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.267542 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eca22f13-1929-48b6-90fd-5d82f58f6017-tmp-dir\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:02.267647 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.267575 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6bjp\" (UniqueName: \"kubernetes.io/projected/6cec11ca-b0a2-4bd4-ae8e-bb2e6ac12df0-kube-api-access-n6bjp\") pod \"ingress-canary-9qqxh\" (UID: \"6cec11ca-b0a2-4bd4-ae8e-bb2e6ac12df0\") " pod="openshift-ingress-canary/ingress-canary-9qqxh" Jul 02 16:55:02.267805 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:02.267674 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:55:02.267805 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.267716 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eca22f13-1929-48b6-90fd-5d82f58f6017-config-volume\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:02.267805 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:02.267757 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls podName:eca22f13-1929-48b6-90fd-5d82f58f6017 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:02.767735772 +0000 UTC m=+36.080609418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls") pod "dns-default-76lqz" (UID: "eca22f13-1929-48b6-90fd-5d82f58f6017") : secret "dns-default-metrics-tls" not found Jul 02 16:55:02.267805 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.267777 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cec11ca-b0a2-4bd4-ae8e-bb2e6ac12df0-cert\") pod \"ingress-canary-9qqxh\" (UID: \"6cec11ca-b0a2-4bd4-ae8e-bb2e6ac12df0\") " pod="openshift-ingress-canary/ingress-canary-9qqxh" Jul 02 16:55:02.267987 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.267895 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eca22f13-1929-48b6-90fd-5d82f58f6017-tmp-dir\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:02.268311 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.268287 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eca22f13-1929-48b6-90fd-5d82f58f6017-config-volume\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:02.272810 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.272779 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cec11ca-b0a2-4bd4-ae8e-bb2e6ac12df0-cert\") pod \"ingress-canary-9qqxh\" (UID: \"6cec11ca-b0a2-4bd4-ae8e-bb2e6ac12df0\") " pod="openshift-ingress-canary/ingress-canary-9qqxh" Jul 02 16:55:02.276141 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.276106 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4l62\" (UniqueName: \"kubernetes.io/projected/eca22f13-1929-48b6-90fd-5d82f58f6017-kube-api-access-h4l62\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:02.276317 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.276297 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6bjp\" (UniqueName: \"kubernetes.io/projected/6cec11ca-b0a2-4bd4-ae8e-bb2e6ac12df0-kube-api-access-n6bjp\") pod \"ingress-canary-9qqxh\" (UID: \"6cec11ca-b0a2-4bd4-ae8e-bb2e6ac12df0\") " pod="openshift-ingress-canary/ingress-canary-9qqxh" Jul 02 16:55:02.367325 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.367286 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9qqxh" Jul 02 16:55:02.771833 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.771803 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:02.771982 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:02.771963 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:55:02.772091 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:02.772079 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls podName:eca22f13-1929-48b6-90fd-5d82f58f6017 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:03.772062382 +0000 UTC m=+37.084936006 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls") pod "dns-default-76lqz" (UID: "eca22f13-1929-48b6-90fd-5d82f58f6017") : secret "dns-default-metrics-tls" not found Jul 02 16:55:02.914084 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:02.914048 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9qqxh"] Jul 02 16:55:02.959174 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:55:02.959131 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cec11ca_b0a2_4bd4_ae8e_bb2e6ac12df0.slice/crio-ba4ea415d57aed8e8dd4036378434c2772b62d0ba13aba02062d40f365725894 WatchSource:0}: Error finding container ba4ea415d57aed8e8dd4036378434c2772b62d0ba13aba02062d40f365725894: Status 404 returned error can't find the container with id ba4ea415d57aed8e8dd4036378434c2772b62d0ba13aba02062d40f365725894 Jul 02 16:55:03.427583 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:03.427296 2556 generic.go:356] "Generic (PLEG): container finished" podID="aa88be7d-b349-4071-8f78-2cdafc2fd875" containerID="8e6286c77b503a1c084419fe9c42d3f41bba7239dcb89a53cabaeda9dc846411" exitCode=0 Jul 02 16:55:03.427583 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:03.427333 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fnsfd" event={"ID":"aa88be7d-b349-4071-8f78-2cdafc2fd875","Type":"ContainerDied","Data":"8e6286c77b503a1c084419fe9c42d3f41bba7239dcb89a53cabaeda9dc846411"} Jul 02 16:55:03.428930 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:03.428897 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9qqxh" event={"ID":"6cec11ca-b0a2-4bd4-ae8e-bb2e6ac12df0","Type":"ContainerStarted","Data":"ba4ea415d57aed8e8dd4036378434c2772b62d0ba13aba02062d40f365725894"} Jul 02 16:55:03.779917 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:03.779884 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:03.780127 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:03.780016 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:55:03.780127 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:03.780074 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls podName:eca22f13-1929-48b6-90fd-5d82f58f6017 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:05.780056344 +0000 UTC m=+39.092929973 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls") pod "dns-default-76lqz" (UID: "eca22f13-1929-48b6-90fd-5d82f58f6017") : secret "dns-default-metrics-tls" not found Jul 02 16:55:04.434277 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:04.434248 2556 generic.go:356] "Generic (PLEG): container finished" podID="aa88be7d-b349-4071-8f78-2cdafc2fd875" containerID="72e9d7cb39a67bef5c217f2019cb7509049b367609a15729bcb5e29afaef21be" exitCode=0 Jul 02 16:55:04.434623 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:04.434304 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fnsfd" event={"ID":"aa88be7d-b349-4071-8f78-2cdafc2fd875","Type":"ContainerDied","Data":"72e9d7cb39a67bef5c217f2019cb7509049b367609a15729bcb5e29afaef21be"} Jul 02 16:55:05.440721 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:05.440678 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fnsfd" event={"ID":"aa88be7d-b349-4071-8f78-2cdafc2fd875","Type":"ContainerStarted","Data":"c09e253de29e4e5f9fa850881ec9f2eacbdf8d150ed931c1f1b1a3d7e7541dbf"} Jul 02 16:55:05.442281 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:05.442256 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9qqxh" event={"ID":"6cec11ca-b0a2-4bd4-ae8e-bb2e6ac12df0","Type":"ContainerStarted","Data":"839c3a7144a5ff0a95a30eebbe2235e5c6236b8a413a3d35549e81d739a61a6f"} Jul 02 16:55:05.464986 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:05.464939 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fnsfd" podStartSLOduration=9.142084019 podStartE2EDuration="38.46492587s" totalImagesPullingTime="29.322841851s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:55:05.463670012 +0000 UTC m=+38.776543658" watchObservedRunningTime="2026-07-02 16:55:05.46492587 +0000 UTC m=+38.777799516" Jul 02 16:55:05.479051 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:05.478997 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9qqxh" podStartSLOduration=2.178856944 podStartE2EDuration="4.478977448s" totalImagesPullingTime="2.300120504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:55:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:55:05.478311984 +0000 UTC m=+38.791185629" watchObservedRunningTime="2026-07-02 16:55:05.478977448 +0000 UTC m=+38.791851094" Jul 02 16:55:05.796950 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:05.796920 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:05.797108 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:05.797031 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:55:05.797108 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:05.797095 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls podName:eca22f13-1929-48b6-90fd-5d82f58f6017 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:09.79708122 +0000 UTC m=+43.109954844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls") pod "dns-default-76lqz" (UID: "eca22f13-1929-48b6-90fd-5d82f58f6017") : secret "dns-default-metrics-tls" not found Jul 02 16:55:09.825788 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:09.825600 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:09.826186 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:09.825763 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:55:09.826186 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:09.825864 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls podName:eca22f13-1929-48b6-90fd-5d82f58f6017 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:17.825849216 +0000 UTC m=+51.138722853 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls") pod "dns-default-76lqz" (UID: "eca22f13-1929-48b6-90fd-5d82f58f6017") : secret "dns-default-metrics-tls" not found Jul 02 16:55:17.881668 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:17.881638 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:17.882117 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:17.881734 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:55:17.882117 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:17.881780 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls podName:eca22f13-1929-48b6-90fd-5d82f58f6017 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:33.881766817 +0000 UTC m=+67.194640440 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls") pod "dns-default-76lqz" (UID: "eca22f13-1929-48b6-90fd-5d82f58f6017") : secret "dns-default-metrics-tls" not found Jul 02 16:55:28.430318 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:28.430290 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qx6zp" Jul 02 16:55:32.886747 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:32.886710 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret\") pod \"global-pull-secret-syncer-76pzw\" (UID: \"3a271183-1ef5-49c2-a88f-7ea46b2b8864\") " pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:55:32.889333 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:32.889315 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 16:55:32.900797 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:32.900779 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a271183-1ef5-49c2-a88f-7ea46b2b8864-original-pull-secret\") pod \"global-pull-secret-syncer-76pzw\" (UID: \"3a271183-1ef5-49c2-a88f-7ea46b2b8864\") " pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:55:32.987278 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:32.987252 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs\") pod \"network-metrics-daemon-bzrql\" (UID: \"6c35723b-8e1e-4682-b963-70e3266a6a71\") " pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:55:32.987387 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:32.987283 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntfj8\" (UniqueName: \"kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8\") pod \"network-check-target-5xcjz\" (UID: \"3e206cf7-e305-45d7-be70-8b153b93bffa\") " pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:55:32.989599 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:32.989585 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 16:55:32.989671 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:32.989653 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 16:55:32.997499 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:32.997472 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 02 16:55:32.997556 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:32.997542 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs podName:6c35723b-8e1e-4682-b963-70e3266a6a71 nodeName:}" failed. No retries permitted until 2026-07-02 16:56:36.997527551 +0000 UTC m=+130.310401189 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs") pod "network-metrics-daemon-bzrql" (UID: "6c35723b-8e1e-4682-b963-70e3266a6a71") : secret "metrics-daemon-secret" not found Jul 02 16:55:32.999690 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:32.999676 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 16:55:33.010411 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:33.010380 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntfj8\" (UniqueName: \"kubernetes.io/projected/3e206cf7-e305-45d7-be70-8b153b93bffa-kube-api-access-ntfj8\") pod \"network-check-target-5xcjz\" (UID: \"3e206cf7-e305-45d7-be70-8b153b93bffa\") " pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:55:33.103510 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:33.103476 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-76pzw" Jul 02 16:55:33.111351 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:33.111328 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-442wp\"" Jul 02 16:55:33.119396 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:33.119377 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:55:33.252409 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:33.252351 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-76pzw"] Jul 02 16:55:33.255072 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:55:33.255044 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a271183_1ef5_49c2_a88f_7ea46b2b8864.slice/crio-a1c4d44456e011483c27d260d06d7eb2bc3319d640f77e24b6769a1efae79c45 WatchSource:0}: Error finding container a1c4d44456e011483c27d260d06d7eb2bc3319d640f77e24b6769a1efae79c45: Status 404 returned error can't find the container with id a1c4d44456e011483c27d260d06d7eb2bc3319d640f77e24b6769a1efae79c45 Jul 02 16:55:33.261800 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:33.261778 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5xcjz"] Jul 02 16:55:33.264618 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:55:33.264595 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e206cf7_e305_45d7_be70_8b153b93bffa.slice/crio-188e999630a9f4a27be7180865185370fce6877eeded50dfd2da7e21f27959f0 WatchSource:0}: Error finding container 188e999630a9f4a27be7180865185370fce6877eeded50dfd2da7e21f27959f0: Status 404 returned error can't find the container with id 188e999630a9f4a27be7180865185370fce6877eeded50dfd2da7e21f27959f0 Jul 02 16:55:33.508382 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:33.508348 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-76pzw" event={"ID":"3a271183-1ef5-49c2-a88f-7ea46b2b8864","Type":"ContainerStarted","Data":"a1c4d44456e011483c27d260d06d7eb2bc3319d640f77e24b6769a1efae79c45"} Jul 02 16:55:33.509221 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:33.509201 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5xcjz" event={"ID":"3e206cf7-e305-45d7-be70-8b153b93bffa","Type":"ContainerStarted","Data":"188e999630a9f4a27be7180865185370fce6877eeded50dfd2da7e21f27959f0"} Jul 02 16:55:33.895822 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:33.895745 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:55:33.896278 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:33.895881 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:55:33.896278 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:55:33.895946 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls podName:eca22f13-1929-48b6-90fd-5d82f58f6017 nodeName:}" failed. No retries permitted until 2026-07-02 16:56:05.895926998 +0000 UTC m=+99.208800636 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls") pod "dns-default-76lqz" (UID: "eca22f13-1929-48b6-90fd-5d82f58f6017") : secret "dns-default-metrics-tls" not found Jul 02 16:55:38.524052 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:38.524017 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5xcjz" event={"ID":"3e206cf7-e305-45d7-be70-8b153b93bffa","Type":"ContainerStarted","Data":"0e5fb70d5091ab37f564de8b56838bfee3d8057624dd66ac3e48cb904e21a814"} Jul 02 16:55:38.524496 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:38.524141 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:55:38.525518 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:38.525495 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-76pzw" event={"ID":"3a271183-1ef5-49c2-a88f-7ea46b2b8864","Type":"ContainerStarted","Data":"0223a2b3d4da0772df76474c0a3d87d59941f45036f1e72a9712525b826dcb0a"} Jul 02 16:55:38.538872 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:38.538822 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5xcjz" podStartSLOduration=66.745149866 podStartE2EDuration="1m11.538806955s" totalImagesPullingTime="4.793657089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:55:38.538352871 +0000 UTC m=+71.851226517" watchObservedRunningTime="2026-07-02 16:55:38.538806955 +0000 UTC m=+71.851680595" Jul 02 16:55:38.551791 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:38.551744 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-76pzw" podStartSLOduration=66.745198128 podStartE2EDuration="1m11.551731916s" totalImagesPullingTime="4.806533788s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:55:38.551547186 +0000 UTC m=+71.864420838" watchObservedRunningTime="2026-07-02 16:55:38.551731916 +0000 UTC m=+71.864605562" Jul 02 16:55:57.393293 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:57.393181 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff8f84c48-9b89f"] Jul 02 16:55:57.395525 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:57.395508 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff8f84c48-9b89f" Jul 02 16:55:57.397973 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:57.397951 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-t2xtb\"" Jul 02 16:55:57.398116 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:57.397951 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 02 16:55:57.398218 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:57.398205 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 02 16:55:57.398999 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:57.398982 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 02 16:55:57.399219 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:57.399207 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 02 16:55:57.405776 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:57.405752 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff8f84c48-9b89f"] Jul 02 16:55:57.450602 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:57.450577 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmj58\" (UniqueName: \"kubernetes.io/projected/e5c5a82b-a161-4d31-a4bf-cdba0bafb024-kube-api-access-nmj58\") pod \"managed-serviceaccount-addon-agent-ff8f84c48-9b89f\" (UID: \"e5c5a82b-a161-4d31-a4bf-cdba0bafb024\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff8f84c48-9b89f" Jul 02 16:55:57.450712 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:57.450612 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5c5a82b-a161-4d31-a4bf-cdba0bafb024-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-ff8f84c48-9b89f\" (UID: \"e5c5a82b-a161-4d31-a4bf-cdba0bafb024\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff8f84c48-9b89f" Jul 02 16:55:57.551274 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:57.551253 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmj58\" (UniqueName: \"kubernetes.io/projected/e5c5a82b-a161-4d31-a4bf-cdba0bafb024-kube-api-access-nmj58\") pod \"managed-serviceaccount-addon-agent-ff8f84c48-9b89f\" (UID: \"e5c5a82b-a161-4d31-a4bf-cdba0bafb024\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff8f84c48-9b89f" Jul 02 16:55:57.551390 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:57.551284 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5c5a82b-a161-4d31-a4bf-cdba0bafb024-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-ff8f84c48-9b89f\" (UID: \"e5c5a82b-a161-4d31-a4bf-cdba0bafb024\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff8f84c48-9b89f" Jul 02 16:55:57.553764 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:57.553742 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5c5a82b-a161-4d31-a4bf-cdba0bafb024-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-ff8f84c48-9b89f\" (UID: \"e5c5a82b-a161-4d31-a4bf-cdba0bafb024\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff8f84c48-9b89f" Jul 02 16:55:57.558759 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:57.558736 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmj58\" (UniqueName: \"kubernetes.io/projected/e5c5a82b-a161-4d31-a4bf-cdba0bafb024-kube-api-access-nmj58\") pod \"managed-serviceaccount-addon-agent-ff8f84c48-9b89f\" (UID: \"e5c5a82b-a161-4d31-a4bf-cdba0bafb024\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff8f84c48-9b89f" Jul 02 16:55:57.713636 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:57.713547 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff8f84c48-9b89f" Jul 02 16:55:57.828189 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:57.828145 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff8f84c48-9b89f"] Jul 02 16:55:57.830939 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:55:57.830913 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5c5a82b_a161_4d31_a4bf_cdba0bafb024.slice/crio-44dba030749425a3ade0a9cd4be9caeb45ac6a298f2979510678da24c88499b5 WatchSource:0}: Error finding container 44dba030749425a3ade0a9cd4be9caeb45ac6a298f2979510678da24c88499b5: Status 404 returned error can't find the container with id 44dba030749425a3ade0a9cd4be9caeb45ac6a298f2979510678da24c88499b5 Jul 02 16:55:58.575949 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:55:58.575906 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff8f84c48-9b89f" event={"ID":"e5c5a82b-a161-4d31-a4bf-cdba0bafb024","Type":"ContainerStarted","Data":"44dba030749425a3ade0a9cd4be9caeb45ac6a298f2979510678da24c88499b5"} Jul 02 16:56:00.581948 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:56:00.581909 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff8f84c48-9b89f" event={"ID":"e5c5a82b-a161-4d31-a4bf-cdba0bafb024","Type":"ContainerStarted","Data":"89e6e28493548dc7d92efebaa6b243741ab219736f05f5880d1227cba84365ed"} Jul 02 16:56:00.598391 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:56:00.598339 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff8f84c48-9b89f" podStartSLOduration=1.441560211 podStartE2EDuration="3.598322117s" totalImagesPullingTime="2.156761906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:55:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:56:00.597669296 +0000 UTC m=+93.910542943" watchObservedRunningTime="2026-07-02 16:56:00.598322117 +0000 UTC m=+93.911195762" Jul 02 16:56:05.906393 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:56:05.906336 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:56:05.906896 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:56:05.906485 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:56:05.906896 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:56:05.906559 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls podName:eca22f13-1929-48b6-90fd-5d82f58f6017 nodeName:}" failed. No retries permitted until 2026-07-02 16:57:09.906542084 +0000 UTC m=+163.219415712 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls") pod "dns-default-76lqz" (UID: "eca22f13-1929-48b6-90fd-5d82f58f6017") : secret "dns-default-metrics-tls" not found Jul 02 16:56:09.530266 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:56:09.530233 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5xcjz" Jul 02 16:56:37.014893 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:56:37.014837 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs\") pod \"network-metrics-daemon-bzrql\" (UID: \"6c35723b-8e1e-4682-b963-70e3266a6a71\") " pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:56:37.015417 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:56:37.014994 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 02 16:56:37.015417 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:56:37.015062 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs podName:6c35723b-8e1e-4682-b963-70e3266a6a71 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:39.015045249 +0000 UTC m=+252.327918878 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs") pod "network-metrics-daemon-bzrql" (UID: "6c35723b-8e1e-4682-b963-70e3266a6a71") : secret "metrics-daemon-secret" not found Jul 02 16:57:05.044733 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:57:05.044677 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-76lqz" podUID="eca22f13-1929-48b6-90fd-5d82f58f6017" Jul 02 16:57:05.196223 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:57:05.196187 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:57:05.731503 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:57:05.731474 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-76lqz" Jul 02 16:57:09.930174 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:57:09.930126 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:57:09.930690 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:57:09.930291 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:57:09.930690 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:57:09.930366 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls podName:eca22f13-1929-48b6-90fd-5d82f58f6017 nodeName:}" failed. No retries permitted until 2026-07-02 16:59:11.930347357 +0000 UTC m=+285.243220981 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls") pod "dns-default-76lqz" (UID: "eca22f13-1929-48b6-90fd-5d82f58f6017") : secret "dns-default-metrics-tls" not found Jul 02 16:57:19.183674 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:57:19.183590 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:58:00.857437 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:00.857400 2556 generic.go:356] "Generic (PLEG): container finished" podID="e5c5a82b-a161-4d31-a4bf-cdba0bafb024" containerID="89e6e28493548dc7d92efebaa6b243741ab219736f05f5880d1227cba84365ed" exitCode=255 Jul 02 16:58:00.857932 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:00.857455 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff8f84c48-9b89f" event={"ID":"e5c5a82b-a161-4d31-a4bf-cdba0bafb024","Type":"ContainerDied","Data":"89e6e28493548dc7d92efebaa6b243741ab219736f05f5880d1227cba84365ed"} Jul 02 16:58:00.863024 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:00.863004 2556 scope.go:117] "RemoveContainer" containerID="89e6e28493548dc7d92efebaa6b243741ab219736f05f5880d1227cba84365ed" Jul 02 16:58:01.861237 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:01.861202 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ff8f84c48-9b89f" event={"ID":"e5c5a82b-a161-4d31-a4bf-cdba0bafb024","Type":"ContainerStarted","Data":"5368e0bd9f8d282d00e62f02a41a44cbfecaaa4f2aa0e61ea11f6c614f182110"} Jul 02 16:58:28.417715 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.417684 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-7kblg"] Jul 02 16:58:28.420291 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.420268 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-7kblg" Jul 02 16:58:28.423094 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.423071 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-hz7l8\"" Jul 02 16:58:28.431800 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.431778 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-7kblg"] Jul 02 16:58:28.513170 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.513121 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g2l5k"] Jul 02 16:58:28.515642 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.515618 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g2l5k" Jul 02 16:58:28.516073 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.516047 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5f67b55555-gzntn"] Jul 02 16:58:28.517876 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.517857 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-tb4kk\"" Jul 02 16:58:28.518211 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.518194 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 02 16:58:28.518514 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.518479 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 02 16:58:28.518673 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.518654 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7"] Jul 02 16:58:28.518794 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.518778 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.520842 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.520826 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ljmhj\"" Jul 02 16:58:28.521024 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.521010 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 16:58:28.521063 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.521031 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 02 16:58:28.521170 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.521137 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 02 16:58:28.521271 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.521257 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 16:58:28.521346 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.521269 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm"] Jul 02 16:58:28.521400 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.521362 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" Jul 02 16:58:28.523910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.523721 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2tjx6\"" Jul 02 16:58:28.523910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.523779 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 02 16:58:28.523910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.523778 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 02 16:58:28.523910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.523819 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-njvpd"] Jul 02 16:58:28.524144 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.524019 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm" Jul 02 16:58:28.524567 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.524551 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 02 16:58:28.526538 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.526519 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:28.527293 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.527271 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-x27qj\"" Jul 02 16:58:28.527376 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.527295 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 02 16:58:28.527376 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.527303 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 02 16:58:28.527376 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.527334 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 02 16:58:28.527619 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.527444 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 02 16:58:28.528425 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.528386 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 02 16:58:28.528908 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.528887 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 02 16:58:28.529411 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.529392 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 02 16:58:28.529513 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.529464 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 02 16:58:28.529513 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.529503 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-hdpht\"" Jul 02 16:58:28.529723 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.529703 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g2l5k"] Jul 02 16:58:28.529800 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.529754 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 02 16:58:28.536013 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.535585 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 02 16:58:28.536013 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.535791 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5f67b55555-gzntn"] Jul 02 16:58:28.536214 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.536145 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7"] Jul 02 16:58:28.536807 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.536787 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm"] Jul 02 16:58:28.545433 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.545410 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5lxp\" (UniqueName: \"kubernetes.io/projected/b013f2c2-3a3a-4285-9f95-36f120e77726-kube-api-access-d5lxp\") pod \"network-check-source-56499946bc-7kblg\" (UID: \"b013f2c2-3a3a-4285-9f95-36f120e77726\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-7kblg" Jul 02 16:58:28.548540 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.548521 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-njvpd"] Jul 02 16:58:28.645832 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.645800 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl572\" (UniqueName: \"kubernetes.io/projected/81ef2a97-c51e-4d07-b26d-083e4e348612-kube-api-access-jl572\") pod \"console-operator-7fdbdf9c99-njvpd\" (UID: \"81ef2a97-c51e-4d07-b26d-083e4e348612\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:28.645997 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.645854 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d1a424d-f615-4619-9406-42979581d953-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.645997 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.645879 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmzm2\" (UniqueName: \"kubernetes.io/projected/9f06edb2-81bf-476f-9321-1a4f0ae93208-kube-api-access-cmzm2\") pod \"cluster-samples-operator-9ddc5ff76-8wpx7\" (UID: \"9f06edb2-81bf-476f-9321-1a4f0ae93208\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" Jul 02 16:58:28.645997 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.645909 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/784b2438-38ab-4c11-a265-ee0b046adfb0-config\") pod \"service-ca-operator-5d7b7dc95c-ghvnm\" (UID: \"784b2438-38ab-4c11-a265-ee0b046adfb0\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm" Jul 02 16:58:28.645997 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.645927 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d1a424d-f615-4619-9406-42979581d953-tmp\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.645997 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.645949 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2872g\" (UniqueName: \"kubernetes.io/projected/4d1a424d-f615-4619-9406-42979581d953-kube-api-access-2872g\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.645997 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.645975 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8wpx7\" (UID: \"9f06edb2-81bf-476f-9321-1a4f0ae93208\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" Jul 02 16:58:28.646228 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.646040 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d1a424d-f615-4619-9406-42979581d953-service-ca-bundle\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.646228 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.646064 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4d1a424d-f615-4619-9406-42979581d953-serving-cert\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.646228 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.646078 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/81ef2a97-c51e-4d07-b26d-083e4e348612-serving-cert\") pod \"console-operator-7fdbdf9c99-njvpd\" (UID: \"81ef2a97-c51e-4d07-b26d-083e4e348612\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:28.646228 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.646106 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/81ef2a97-c51e-4d07-b26d-083e4e348612-trusted-ca\") pod \"console-operator-7fdbdf9c99-njvpd\" (UID: \"81ef2a97-c51e-4d07-b26d-083e4e348612\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:28.646228 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.646132 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5lxp\" (UniqueName: \"kubernetes.io/projected/b013f2c2-3a3a-4285-9f95-36f120e77726-kube-api-access-d5lxp\") pod \"network-check-source-56499946bc-7kblg\" (UID: \"b013f2c2-3a3a-4285-9f95-36f120e77726\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-7kblg" Jul 02 16:58:28.646228 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.646147 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4d1a424d-f615-4619-9406-42979581d953-snapshots\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.646228 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.646207 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wbnr\" (UniqueName: \"kubernetes.io/projected/784b2438-38ab-4c11-a265-ee0b046adfb0-kube-api-access-4wbnr\") pod \"service-ca-operator-5d7b7dc95c-ghvnm\" (UID: \"784b2438-38ab-4c11-a265-ee0b046adfb0\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm" Jul 02 16:58:28.646452 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.646231 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/81ef2a97-c51e-4d07-b26d-083e4e348612-config\") pod \"console-operator-7fdbdf9c99-njvpd\" (UID: \"81ef2a97-c51e-4d07-b26d-083e4e348612\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:28.646452 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.646253 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9ltt\" (UniqueName: \"kubernetes.io/projected/edc0b924-909a-4f9a-87d6-1c3d0426cd3f-kube-api-access-w9ltt\") pod \"volume-data-source-validator-69d9f679db-g2l5k\" (UID: \"edc0b924-909a-4f9a-87d6-1c3d0426cd3f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g2l5k" Jul 02 16:58:28.646452 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.646281 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/784b2438-38ab-4c11-a265-ee0b046adfb0-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-ghvnm\" (UID: \"784b2438-38ab-4c11-a265-ee0b046adfb0\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm" Jul 02 16:58:28.654073 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.654039 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5lxp\" (UniqueName: \"kubernetes.io/projected/b013f2c2-3a3a-4285-9f95-36f120e77726-kube-api-access-d5lxp\") pod \"network-check-source-56499946bc-7kblg\" (UID: \"b013f2c2-3a3a-4285-9f95-36f120e77726\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-7kblg" Jul 02 16:58:28.729028 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.728967 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-7kblg" Jul 02 16:58:28.747131 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.747109 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4d1a424d-f615-4619-9406-42979581d953-serving-cert\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.747230 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.747135 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/81ef2a97-c51e-4d07-b26d-083e4e348612-serving-cert\") pod \"console-operator-7fdbdf9c99-njvpd\" (UID: \"81ef2a97-c51e-4d07-b26d-083e4e348612\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:28.747230 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.747154 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/81ef2a97-c51e-4d07-b26d-083e4e348612-trusted-ca\") pod \"console-operator-7fdbdf9c99-njvpd\" (UID: \"81ef2a97-c51e-4d07-b26d-083e4e348612\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:28.747230 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.747191 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4d1a424d-f615-4619-9406-42979581d953-snapshots\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.747230 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.747216 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wbnr\" (UniqueName: \"kubernetes.io/projected/784b2438-38ab-4c11-a265-ee0b046adfb0-kube-api-access-4wbnr\") pod \"service-ca-operator-5d7b7dc95c-ghvnm\" (UID: \"784b2438-38ab-4c11-a265-ee0b046adfb0\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm" Jul 02 16:58:28.747437 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.747338 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/81ef2a97-c51e-4d07-b26d-083e4e348612-config\") pod \"console-operator-7fdbdf9c99-njvpd\" (UID: \"81ef2a97-c51e-4d07-b26d-083e4e348612\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:28.747437 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.747382 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9ltt\" (UniqueName: \"kubernetes.io/projected/edc0b924-909a-4f9a-87d6-1c3d0426cd3f-kube-api-access-w9ltt\") pod \"volume-data-source-validator-69d9f679db-g2l5k\" (UID: \"edc0b924-909a-4f9a-87d6-1c3d0426cd3f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g2l5k" Jul 02 16:58:28.747437 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.747425 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/784b2438-38ab-4c11-a265-ee0b046adfb0-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-ghvnm\" (UID: \"784b2438-38ab-4c11-a265-ee0b046adfb0\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm" Jul 02 16:58:28.748598 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.747464 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl572\" (UniqueName: \"kubernetes.io/projected/81ef2a97-c51e-4d07-b26d-083e4e348612-kube-api-access-jl572\") pod \"console-operator-7fdbdf9c99-njvpd\" (UID: \"81ef2a97-c51e-4d07-b26d-083e4e348612\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:28.748598 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.747513 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d1a424d-f615-4619-9406-42979581d953-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.748598 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.747537 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmzm2\" (UniqueName: \"kubernetes.io/projected/9f06edb2-81bf-476f-9321-1a4f0ae93208-kube-api-access-cmzm2\") pod \"cluster-samples-operator-9ddc5ff76-8wpx7\" (UID: \"9f06edb2-81bf-476f-9321-1a4f0ae93208\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" Jul 02 16:58:28.748598 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.747874 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/784b2438-38ab-4c11-a265-ee0b046adfb0-config\") pod \"service-ca-operator-5d7b7dc95c-ghvnm\" (UID: \"784b2438-38ab-4c11-a265-ee0b046adfb0\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm" Jul 02 16:58:28.748598 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.747901 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d1a424d-f615-4619-9406-42979581d953-tmp\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.748598 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.747928 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2872g\" (UniqueName: \"kubernetes.io/projected/4d1a424d-f615-4619-9406-42979581d953-kube-api-access-2872g\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.748598 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.747950 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8wpx7\" (UID: \"9f06edb2-81bf-476f-9321-1a4f0ae93208\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" Jul 02 16:58:28.748598 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.747987 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d1a424d-f615-4619-9406-42979581d953-service-ca-bundle\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.748598 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.748079 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4d1a424d-f615-4619-9406-42979581d953-snapshots\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.748598 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.748217 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/81ef2a97-c51e-4d07-b26d-083e4e348612-config\") pod \"console-operator-7fdbdf9c99-njvpd\" (UID: \"81ef2a97-c51e-4d07-b26d-083e4e348612\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:28.748598 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.748278 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/81ef2a97-c51e-4d07-b26d-083e4e348612-trusted-ca\") pod \"console-operator-7fdbdf9c99-njvpd\" (UID: \"81ef2a97-c51e-4d07-b26d-083e4e348612\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:28.748598 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.748342 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d1a424d-f615-4619-9406-42979581d953-tmp\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.748598 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:28.748445 2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 16:58:28.748598 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.748501 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d1a424d-f615-4619-9406-42979581d953-service-ca-bundle\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.748598 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:28.748541 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls podName:9f06edb2-81bf-476f-9321-1a4f0ae93208 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:29.248520634 +0000 UTC m=+242.561394278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-8wpx7" (UID: "9f06edb2-81bf-476f-9321-1a4f0ae93208") : secret "samples-operator-tls" not found Jul 02 16:58:28.748598 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.748588 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d1a424d-f615-4619-9406-42979581d953-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.749128 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.748804 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/784b2438-38ab-4c11-a265-ee0b046adfb0-config\") pod \"service-ca-operator-5d7b7dc95c-ghvnm\" (UID: \"784b2438-38ab-4c11-a265-ee0b046adfb0\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm" Jul 02 16:58:28.749977 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.749959 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4d1a424d-f615-4619-9406-42979581d953-serving-cert\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.750340 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.750317 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/784b2438-38ab-4c11-a265-ee0b046adfb0-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-ghvnm\" (UID: \"784b2438-38ab-4c11-a265-ee0b046adfb0\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm" Jul 02 16:58:28.750425 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.750366 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/81ef2a97-c51e-4d07-b26d-083e4e348612-serving-cert\") pod \"console-operator-7fdbdf9c99-njvpd\" (UID: \"81ef2a97-c51e-4d07-b26d-083e4e348612\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:28.755531 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.755488 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl572\" (UniqueName: \"kubernetes.io/projected/81ef2a97-c51e-4d07-b26d-083e4e348612-kube-api-access-jl572\") pod \"console-operator-7fdbdf9c99-njvpd\" (UID: \"81ef2a97-c51e-4d07-b26d-083e4e348612\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:28.755984 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.755964 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wbnr\" (UniqueName: \"kubernetes.io/projected/784b2438-38ab-4c11-a265-ee0b046adfb0-kube-api-access-4wbnr\") pod \"service-ca-operator-5d7b7dc95c-ghvnm\" (UID: \"784b2438-38ab-4c11-a265-ee0b046adfb0\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm" Jul 02 16:58:28.756148 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.756121 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9ltt\" (UniqueName: \"kubernetes.io/projected/edc0b924-909a-4f9a-87d6-1c3d0426cd3f-kube-api-access-w9ltt\") pod \"volume-data-source-validator-69d9f679db-g2l5k\" (UID: \"edc0b924-909a-4f9a-87d6-1c3d0426cd3f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g2l5k" Jul 02 16:58:28.756604 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.756583 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmzm2\" (UniqueName: \"kubernetes.io/projected/9f06edb2-81bf-476f-9321-1a4f0ae93208-kube-api-access-cmzm2\") pod \"cluster-samples-operator-9ddc5ff76-8wpx7\" (UID: \"9f06edb2-81bf-476f-9321-1a4f0ae93208\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" Jul 02 16:58:28.757149 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.757128 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2872g\" (UniqueName: \"kubernetes.io/projected/4d1a424d-f615-4619-9406-42979581d953-kube-api-access-2872g\") pod \"insights-operator-5f67b55555-gzntn\" (UID: \"4d1a424d-f615-4619-9406-42979581d953\") " pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.826584 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.826554 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g2l5k" Jul 02 16:58:28.836692 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.836668 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5f67b55555-gzntn" Jul 02 16:58:28.847818 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.847693 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm" Jul 02 16:58:28.849204 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.849182 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-7kblg"] Jul 02 16:58:28.852668 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:58:28.852641 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb013f2c2_3a3a_4285_9f95_36f120e77726.slice/crio-042f27fa8706746fbaefd2835c597a92e39f75fbc3025fcf837c6f487fe66243 WatchSource:0}: Error finding container 042f27fa8706746fbaefd2835c597a92e39f75fbc3025fcf837c6f487fe66243: Status 404 returned error can't find the container with id 042f27fa8706746fbaefd2835c597a92e39f75fbc3025fcf837c6f487fe66243 Jul 02 16:58:28.853759 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.853665 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:28.930997 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.930905 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-7kblg" event={"ID":"b013f2c2-3a3a-4285-9f95-36f120e77726","Type":"ContainerStarted","Data":"042f27fa8706746fbaefd2835c597a92e39f75fbc3025fcf837c6f487fe66243"} Jul 02 16:58:28.984039 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:28.983961 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g2l5k"] Jul 02 16:58:28.986402 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:58:28.986374 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedc0b924_909a_4f9a_87d6_1c3d0426cd3f.slice/crio-4f62f6610a4c3f02a3536f01e58c9779fd8dae85eca8168b7c513772d7f76850 WatchSource:0}: Error finding container 4f62f6610a4c3f02a3536f01e58c9779fd8dae85eca8168b7c513772d7f76850: Status 404 returned error can't find the container with id 4f62f6610a4c3f02a3536f01e58c9779fd8dae85eca8168b7c513772d7f76850 Jul 02 16:58:29.001483 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:29.001457 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5f67b55555-gzntn"] Jul 02 16:58:29.006134 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:58:29.006111 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d1a424d_f615_4619_9406_42979581d953.slice/crio-274837eb227db9be6a1c9d4a354f01487f7489e15d8ebedf654a807b7f7e57ac WatchSource:0}: Error finding container 274837eb227db9be6a1c9d4a354f01487f7489e15d8ebedf654a807b7f7e57ac: Status 404 returned error can't find the container with id 274837eb227db9be6a1c9d4a354f01487f7489e15d8ebedf654a807b7f7e57ac Jul 02 16:58:29.026472 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:29.026451 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm"] Jul 02 16:58:29.028829 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:58:29.028802 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod784b2438_38ab_4c11_a265_ee0b046adfb0.slice/crio-ba9b2a19c13b1d00115f6c9cd3018b6bee91d9e549226253edcaab269aa46085 WatchSource:0}: Error finding container ba9b2a19c13b1d00115f6c9cd3018b6bee91d9e549226253edcaab269aa46085: Status 404 returned error can't find the container with id ba9b2a19c13b1d00115f6c9cd3018b6bee91d9e549226253edcaab269aa46085 Jul 02 16:58:29.039728 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:29.039708 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-njvpd"] Jul 02 16:58:29.041095 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:58:29.041070 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81ef2a97_c51e_4d07_b26d_083e4e348612.slice/crio-230d08e15ee3364c0cb87731611975a6586f914d0bbb6b19c6cb0524aec0364b WatchSource:0}: Error finding container 230d08e15ee3364c0cb87731611975a6586f914d0bbb6b19c6cb0524aec0364b: Status 404 returned error can't find the container with id 230d08e15ee3364c0cb87731611975a6586f914d0bbb6b19c6cb0524aec0364b Jul 02 16:58:29.253226 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:29.253203 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8wpx7\" (UID: \"9f06edb2-81bf-476f-9321-1a4f0ae93208\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" Jul 02 16:58:29.253387 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:29.253373 2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 16:58:29.253438 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:29.253428 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls podName:9f06edb2-81bf-476f-9321-1a4f0ae93208 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:30.253412499 +0000 UTC m=+243.566286123 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-8wpx7" (UID: "9f06edb2-81bf-476f-9321-1a4f0ae93208") : secret "samples-operator-tls" not found Jul 02 16:58:29.940211 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:29.940094 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm" event={"ID":"784b2438-38ab-4c11-a265-ee0b046adfb0","Type":"ContainerStarted","Data":"ba9b2a19c13b1d00115f6c9cd3018b6bee91d9e549226253edcaab269aa46085"} Jul 02 16:58:29.944014 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:29.943879 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-7kblg" event={"ID":"b013f2c2-3a3a-4285-9f95-36f120e77726","Type":"ContainerStarted","Data":"2b8f5cb34f0acfb143a2d8dbfbb1702e7cfdaad558b2c9319e3b3e2a68a59215"} Jul 02 16:58:29.946087 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:29.946058 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-gzntn" event={"ID":"4d1a424d-f615-4619-9406-42979581d953","Type":"ContainerStarted","Data":"274837eb227db9be6a1c9d4a354f01487f7489e15d8ebedf654a807b7f7e57ac"} Jul 02 16:58:29.951575 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:29.951550 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g2l5k" event={"ID":"edc0b924-909a-4f9a-87d6-1c3d0426cd3f","Type":"ContainerStarted","Data":"4f62f6610a4c3f02a3536f01e58c9779fd8dae85eca8168b7c513772d7f76850"} Jul 02 16:58:29.952773 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:29.952750 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" event={"ID":"81ef2a97-c51e-4d07-b26d-083e4e348612","Type":"ContainerStarted","Data":"230d08e15ee3364c0cb87731611975a6586f914d0bbb6b19c6cb0524aec0364b"} Jul 02 16:58:29.961902 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:29.961744 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-56499946bc-7kblg" podStartSLOduration=1.961729054 podStartE2EDuration="1.961729054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:58:29.960488871 +0000 UTC m=+243.273362518" watchObservedRunningTime="2026-07-02 16:58:29.961729054 +0000 UTC m=+243.274602702" Jul 02 16:58:30.263874 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:30.263777 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8wpx7\" (UID: \"9f06edb2-81bf-476f-9321-1a4f0ae93208\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" Jul 02 16:58:30.264083 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:30.263980 2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 16:58:30.264083 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:30.264042 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls podName:9f06edb2-81bf-476f-9321-1a4f0ae93208 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:32.264023416 +0000 UTC m=+245.576897044 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-8wpx7" (UID: "9f06edb2-81bf-476f-9321-1a4f0ae93208") : secret "samples-operator-tls" not found Jul 02 16:58:30.958219 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:30.958102 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g2l5k" event={"ID":"edc0b924-909a-4f9a-87d6-1c3d0426cd3f","Type":"ContainerStarted","Data":"ebf06813942c18fae11a29d2ebff8d7bc6046571075d9ca9eb6c52422d97ad20"} Jul 02 16:58:30.974460 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:30.974408 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g2l5k" podStartSLOduration=1.475693144 podStartE2EDuration="2.974388628s" totalImagesPullingTime="1.498695484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:58:30.971985661 +0000 UTC m=+244.284859308" watchObservedRunningTime="2026-07-02 16:58:30.974388628 +0000 UTC m=+244.287262275" Jul 02 16:58:32.278147 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:32.278085 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8wpx7\" (UID: \"9f06edb2-81bf-476f-9321-1a4f0ae93208\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" Jul 02 16:58:32.278642 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:32.278234 2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 16:58:32.278642 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:32.278322 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls podName:9f06edb2-81bf-476f-9321-1a4f0ae93208 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:36.27830036 +0000 UTC m=+249.591173992 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-8wpx7" (UID: "9f06edb2-81bf-476f-9321-1a4f0ae93208") : secret "samples-operator-tls" not found Jul 02 16:58:32.966404 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:32.966326 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/0.log" Jul 02 16:58:32.966404 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:32.966369 2556 generic.go:356] "Generic (PLEG): container finished" podID="81ef2a97-c51e-4d07-b26d-083e4e348612" containerID="12f63566e25a130e3aca6c3061c4a1c81ec31a87204e4fb69c569c5154a7b8b5" exitCode=255 Jul 02 16:58:32.966626 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:32.966456 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" event={"ID":"81ef2a97-c51e-4d07-b26d-083e4e348612","Type":"ContainerDied","Data":"12f63566e25a130e3aca6c3061c4a1c81ec31a87204e4fb69c569c5154a7b8b5"} Jul 02 16:58:32.966776 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:32.966754 2556 scope.go:117] "RemoveContainer" containerID="12f63566e25a130e3aca6c3061c4a1c81ec31a87204e4fb69c569c5154a7b8b5" Jul 02 16:58:32.968134 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:32.968108 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm" event={"ID":"784b2438-38ab-4c11-a265-ee0b046adfb0","Type":"ContainerStarted","Data":"9c22dab5577ab80c1747493eeb6fa96b2196a5a6166dfc0332225f80b01cd072"} Jul 02 16:58:32.969582 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:32.969562 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-gzntn" event={"ID":"4d1a424d-f615-4619-9406-42979581d953","Type":"ContainerStarted","Data":"0808bc93f81236a42d18a49b3a81df86a552b69ccbcfe2a8517de6e57ec77b9c"} Jul 02 16:58:32.982274 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:32.982255 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-rq4sf"] Jul 02 16:58:32.986515 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:32.986491 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-rq4sf" Jul 02 16:58:32.988909 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:32.988885 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 02 16:58:32.989011 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:32.988932 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 02 16:58:32.989210 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:32.989194 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-b5fhp\"" Jul 02 16:58:32.994751 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:32.994730 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-rq4sf"] Jul 02 16:58:33.006119 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.006080 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm" podStartSLOduration=1.645168869 podStartE2EDuration="5.006066207s" totalImagesPullingTime="3.360897338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:58:33.005073738 +0000 UTC m=+246.317947387" watchObservedRunningTime="2026-07-02 16:58:33.006066207 +0000 UTC m=+246.318939852" Jul 02 16:58:33.043020 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.042977 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5f67b55555-gzntn" podStartSLOduration=1.658460228 podStartE2EDuration="5.042957685s" totalImagesPullingTime="3.384497457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:58:33.042185369 +0000 UTC m=+246.355059016" watchObservedRunningTime="2026-07-02 16:58:33.042957685 +0000 UTC m=+246.355831334" Jul 02 16:58:33.085595 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.085564 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tztct\" (UniqueName: \"kubernetes.io/projected/570da710-50a2-4118-bb22-ea95e478156f-kube-api-access-tztct\") pod \"migrator-786f9fbc5-rq4sf\" (UID: \"570da710-50a2-4118-bb22-ea95e478156f\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-rq4sf" Jul 02 16:58:33.186311 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.186278 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tztct\" (UniqueName: \"kubernetes.io/projected/570da710-50a2-4118-bb22-ea95e478156f-kube-api-access-tztct\") pod \"migrator-786f9fbc5-rq4sf\" (UID: \"570da710-50a2-4118-bb22-ea95e478156f\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-rq4sf" Jul 02 16:58:33.194995 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.194967 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tztct\" (UniqueName: \"kubernetes.io/projected/570da710-50a2-4118-bb22-ea95e478156f-kube-api-access-tztct\") pod \"migrator-786f9fbc5-rq4sf\" (UID: \"570da710-50a2-4118-bb22-ea95e478156f\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-rq4sf" Jul 02 16:58:33.308122 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.308094 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-rq4sf" Jul 02 16:58:33.434684 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.434655 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-rq4sf"] Jul 02 16:58:33.438609 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:58:33.438586 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod570da710_50a2_4118_bb22_ea95e478156f.slice/crio-9d3b3b6854be26d2aee7b0f6e7189103ea530aae70e410959b5c1b4a688bac04 WatchSource:0}: Error finding container 9d3b3b6854be26d2aee7b0f6e7189103ea530aae70e410959b5c1b4a688bac04: Status 404 returned error can't find the container with id 9d3b3b6854be26d2aee7b0f6e7189103ea530aae70e410959b5c1b4a688bac04 Jul 02 16:58:33.677683 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.677599 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4l2sg"] Jul 02 16:58:33.681914 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.681893 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:33.684740 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.684719 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 16:58:33.684954 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.684781 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 16:58:33.685062 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.684804 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pdlkg\"" Jul 02 16:58:33.692116 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.692093 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4l2sg"] Jul 02 16:58:33.790424 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.790401 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km58z\" (UniqueName: \"kubernetes.io/projected/82bb767c-600e-4125-bf49-254356619b37-kube-api-access-km58z\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:33.790541 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.790449 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/82bb767c-600e-4125-bf49-254356619b37-crio-socket\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:33.790598 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.790563 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:33.790640 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.790608 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/82bb767c-600e-4125-bf49-254356619b37-data-volume\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:33.790695 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.790678 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:33.891539 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.891508 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:33.891679 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.891550 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km58z\" (UniqueName: \"kubernetes.io/projected/82bb767c-600e-4125-bf49-254356619b37-kube-api-access-km58z\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:33.891679 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.891598 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/82bb767c-600e-4125-bf49-254356619b37-crio-socket\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:33.891679 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:33.891640 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 16:58:33.891679 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.891676 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/82bb767c-600e-4125-bf49-254356619b37-crio-socket\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:33.891878 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:33.891701 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm podName:82bb767c-600e-4125-bf49-254356619b37 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:34.391685302 +0000 UTC m=+247.704558950 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4l2sg" (UID: "82bb767c-600e-4125-bf49-254356619b37") : configmap "kube-rbac-proxy" not found Jul 02 16:58:33.891878 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.891749 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:33.891878 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.891779 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/82bb767c-600e-4125-bf49-254356619b37-data-volume\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:33.891878 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:33.891871 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 16:58:33.892039 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:33.891929 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls podName:82bb767c-600e-4125-bf49-254356619b37 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:34.391913142 +0000 UTC m=+247.704786772 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4l2sg" (UID: "82bb767c-600e-4125-bf49-254356619b37") : secret "insights-runtime-extractor-tls" not found Jul 02 16:58:33.892039 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.892030 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/82bb767c-600e-4125-bf49-254356619b37-data-volume\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:33.900948 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.900922 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-km58z\" (UniqueName: \"kubernetes.io/projected/82bb767c-600e-4125-bf49-254356619b37-kube-api-access-km58z\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:33.973435 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.973371 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-rq4sf" event={"ID":"570da710-50a2-4118-bb22-ea95e478156f","Type":"ContainerStarted","Data":"9d3b3b6854be26d2aee7b0f6e7189103ea530aae70e410959b5c1b4a688bac04"} Jul 02 16:58:33.974781 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.974764 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/1.log" Jul 02 16:58:33.975149 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.975135 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/0.log" Jul 02 16:58:33.975233 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.975180 2556 generic.go:356] "Generic (PLEG): container finished" podID="81ef2a97-c51e-4d07-b26d-083e4e348612" containerID="52202971f34e8e3d358cc4116d147a1f54c1a1b2a27096e2a8a443eae21ef6fc" exitCode=255 Jul 02 16:58:33.975290 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.975245 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" event={"ID":"81ef2a97-c51e-4d07-b26d-083e4e348612","Type":"ContainerDied","Data":"52202971f34e8e3d358cc4116d147a1f54c1a1b2a27096e2a8a443eae21ef6fc"} Jul 02 16:58:33.975329 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.975290 2556 scope.go:117] "RemoveContainer" containerID="12f63566e25a130e3aca6c3061c4a1c81ec31a87204e4fb69c569c5154a7b8b5" Jul 02 16:58:33.975478 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:33.975451 2556 scope.go:117] "RemoveContainer" containerID="52202971f34e8e3d358cc4116d147a1f54c1a1b2a27096e2a8a443eae21ef6fc" Jul 02 16:58:33.975653 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:33.975633 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-njvpd_openshift-console-operator(81ef2a97-c51e-4d07-b26d-083e4e348612)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" podUID="81ef2a97-c51e-4d07-b26d-083e4e348612" Jul 02 16:58:34.395154 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:34.395118 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:34.395520 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:34.395228 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:34.395520 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:34.395280 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 16:58:34.395520 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:34.395346 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 16:58:34.395520 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:34.395360 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls podName:82bb767c-600e-4125-bf49-254356619b37 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:35.39533931 +0000 UTC m=+248.708212949 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4l2sg" (UID: "82bb767c-600e-4125-bf49-254356619b37") : secret "insights-runtime-extractor-tls" not found Jul 02 16:58:34.395520 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:34.395382 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm podName:82bb767c-600e-4125-bf49-254356619b37 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:35.395371881 +0000 UTC m=+248.708245515 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4l2sg" (UID: "82bb767c-600e-4125-bf49-254356619b37") : configmap "kube-rbac-proxy" not found Jul 02 16:58:34.979414 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:34.979333 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/1.log" Jul 02 16:58:34.979712 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:34.979686 2556 scope.go:117] "RemoveContainer" containerID="52202971f34e8e3d358cc4116d147a1f54c1a1b2a27096e2a8a443eae21ef6fc" Jul 02 16:58:34.979872 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:34.979854 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-njvpd_openshift-console-operator(81ef2a97-c51e-4d07-b26d-083e4e348612)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" podUID="81ef2a97-c51e-4d07-b26d-083e4e348612" Jul 02 16:58:34.981032 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:34.981010 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-rq4sf" event={"ID":"570da710-50a2-4118-bb22-ea95e478156f","Type":"ContainerStarted","Data":"5dfd58c6a1a10e398b22cc06c57f33c252271f8a5932472513f14dd487951819"} Jul 02 16:58:34.981118 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:34.981051 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-rq4sf" event={"ID":"570da710-50a2-4118-bb22-ea95e478156f","Type":"ContainerStarted","Data":"7b12d46e9547f8f1b43bff69255da8f66601d6d05ff89a731a5d6b7fd94d43e6"} Jul 02 16:58:35.011656 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:35.011613 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-rq4sf" podStartSLOduration=1.8120729450000002 podStartE2EDuration="3.011600439s" totalImagesPullingTime="1.199527494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:58:35.010431179 +0000 UTC m=+248.323304825" watchObservedRunningTime="2026-07-02 16:58:35.011600439 +0000 UTC m=+248.324474085" Jul 02 16:58:35.402809 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:35.402778 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:35.403245 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:35.402824 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:35.403245 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:35.402910 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 16:58:35.403245 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:35.402947 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 16:58:35.403245 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:35.402962 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm podName:82bb767c-600e-4125-bf49-254356619b37 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:37.402950155 +0000 UTC m=+250.715823782 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4l2sg" (UID: "82bb767c-600e-4125-bf49-254356619b37") : configmap "kube-rbac-proxy" not found Jul 02 16:58:35.403245 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:35.403012 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls podName:82bb767c-600e-4125-bf49-254356619b37 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:37.40299292 +0000 UTC m=+250.715866558 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4l2sg" (UID: "82bb767c-600e-4125-bf49-254356619b37") : secret "insights-runtime-extractor-tls" not found Jul 02 16:58:36.257117 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:36.257085 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6c5gj_28783fce-657d-4461-84ca-ae848eb09b15/dns-node-resolver/0.log" Jul 02 16:58:36.310397 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:36.310370 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8wpx7\" (UID: \"9f06edb2-81bf-476f-9321-1a4f0ae93208\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" Jul 02 16:58:36.310537 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:36.310462 2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 16:58:36.310537 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:36.310518 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls podName:9f06edb2-81bf-476f-9321-1a4f0ae93208 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:44.310497787 +0000 UTC m=+257.623371411 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-8wpx7" (UID: "9f06edb2-81bf-476f-9321-1a4f0ae93208") : secret "samples-operator-tls" not found Jul 02 16:58:37.258397 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:37.258369 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-clww4_dfb32fb2-9a93-4405-be31-8c26f66032e1/node-ca/0.log" Jul 02 16:58:37.417341 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:37.417308 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:37.417494 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:37.417398 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:37.417494 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:37.417417 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 16:58:37.417494 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:37.417476 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm podName:82bb767c-600e-4125-bf49-254356619b37 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:41.417462439 +0000 UTC m=+254.730336064 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4l2sg" (UID: "82bb767c-600e-4125-bf49-254356619b37") : configmap "kube-rbac-proxy" not found Jul 02 16:58:37.417704 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:37.417509 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 16:58:37.417704 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:37.417566 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls podName:82bb767c-600e-4125-bf49-254356619b37 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:41.41754961 +0000 UTC m=+254.730423266 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4l2sg" (UID: "82bb767c-600e-4125-bf49-254356619b37") : secret "insights-runtime-extractor-tls" not found Jul 02 16:58:38.057333 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:38.057302 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9qqxh_6cec11ca-b0a2-4bd4-ae8e-bb2e6ac12df0/serve-healthcheck-canary/0.log" Jul 02 16:58:38.657463 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:38.657434 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-rq4sf_570da710-50a2-4118-bb22-ea95e478156f/migrator/0.log" Jul 02 16:58:38.854740 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:38.854719 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:38.854893 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:38.854800 2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:38.855069 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:38.855056 2556 scope.go:117] "RemoveContainer" containerID="52202971f34e8e3d358cc4116d147a1f54c1a1b2a27096e2a8a443eae21ef6fc" Jul 02 16:58:38.855241 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:38.855224 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-njvpd_openshift-console-operator(81ef2a97-c51e-4d07-b26d-083e4e348612)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" podUID="81ef2a97-c51e-4d07-b26d-083e4e348612" Jul 02 16:58:38.856481 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:38.856449 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-rq4sf_570da710-50a2-4118-bb22-ea95e478156f/graceful-termination/0.log" Jul 02 16:58:38.994807 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:38.994786 2556 scope.go:117] "RemoveContainer" containerID="52202971f34e8e3d358cc4116d147a1f54c1a1b2a27096e2a8a443eae21ef6fc" Jul 02 16:58:38.994936 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:38.994913 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-njvpd_openshift-console-operator(81ef2a97-c51e-4d07-b26d-083e4e348612)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" podUID="81ef2a97-c51e-4d07-b26d-083e4e348612" Jul 02 16:58:39.028631 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:39.028605 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs\") pod \"network-metrics-daemon-bzrql\" (UID: \"6c35723b-8e1e-4682-b963-70e3266a6a71\") " pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:58:39.028714 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:39.028701 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 02 16:58:39.028763 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:39.028754 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs podName:6c35723b-8e1e-4682-b963-70e3266a6a71 nodeName:}" failed. No retries permitted until 2026-07-02 17:00:41.028738503 +0000 UTC m=+374.341612127 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs") pod "network-metrics-daemon-bzrql" (UID: "6c35723b-8e1e-4682-b963-70e3266a6a71") : secret "metrics-daemon-secret" not found Jul 02 16:58:41.447027 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:41.446997 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:41.447521 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:41.447058 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:41.447521 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:41.447147 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 16:58:41.447521 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:41.447178 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 16:58:41.447521 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:41.447248 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm podName:82bb767c-600e-4125-bf49-254356619b37 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:49.447229232 +0000 UTC m=+262.760102857 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4l2sg" (UID: "82bb767c-600e-4125-bf49-254356619b37") : configmap "kube-rbac-proxy" not found Jul 02 16:58:41.447521 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:41.447267 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls podName:82bb767c-600e-4125-bf49-254356619b37 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:49.447258243 +0000 UTC m=+262.760131868 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4l2sg" (UID: "82bb767c-600e-4125-bf49-254356619b37") : secret "insights-runtime-extractor-tls" not found Jul 02 16:58:44.371490 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:44.371459 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8wpx7\" (UID: \"9f06edb2-81bf-476f-9321-1a4f0ae93208\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" Jul 02 16:58:44.373675 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:44.373651 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f06edb2-81bf-476f-9321-1a4f0ae93208-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-8wpx7\" (UID: \"9f06edb2-81bf-476f-9321-1a4f0ae93208\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" Jul 02 16:58:44.442931 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:44.442908 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" Jul 02 16:58:44.561182 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:44.561138 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7"] Jul 02 16:58:45.014123 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:45.014084 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" event={"ID":"9f06edb2-81bf-476f-9321-1a4f0ae93208","Type":"ContainerStarted","Data":"4a807bd20f7b7eb8d56684faccbe5c505ebd08f8fc70c920bb58cb72650fb8a2"} Jul 02 16:58:47.022473 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:47.022436 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" event={"ID":"9f06edb2-81bf-476f-9321-1a4f0ae93208","Type":"ContainerStarted","Data":"eb52ea2c36f35dfcc0d02694019692b99243f3e1acc08e33b70851888060285c"} Jul 02 16:58:47.022867 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:47.022483 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" event={"ID":"9f06edb2-81bf-476f-9321-1a4f0ae93208","Type":"ContainerStarted","Data":"4592bda598226d5e3bb1bc0039b58bf388e93df8e6de3b0262a735091d3e20aa"} Jul 02 16:58:47.041225 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:47.041155 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-8wpx7" podStartSLOduration=17.148010877 podStartE2EDuration="19.041139946s" totalImagesPullingTime="1.893129069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:58:47.040756347 +0000 UTC m=+260.353630016" watchObservedRunningTime="2026-07-02 16:58:47.041139946 +0000 UTC m=+260.354013649" Jul 02 16:58:49.511324 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:49.511239 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:49.511324 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:49.511287 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:49.511798 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:49.511404 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 16:58:49.511798 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:49.511485 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls podName:82bb767c-600e-4125-bf49-254356619b37 nodeName:}" failed. No retries permitted until 2026-07-02 16:59:05.511465881 +0000 UTC m=+278.824339506 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4l2sg" (UID: "82bb767c-600e-4125-bf49-254356619b37") : secret "insights-runtime-extractor-tls" not found Jul 02 16:58:49.511798 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:49.511713 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82bb767c-600e-4125-bf49-254356619b37-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:58:53.184061 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:53.184033 2556 scope.go:117] "RemoveContainer" containerID="52202971f34e8e3d358cc4116d147a1f54c1a1b2a27096e2a8a443eae21ef6fc" Jul 02 16:58:54.045350 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:54.045324 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 16:58:54.045747 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:54.045732 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/1.log" Jul 02 16:58:54.045816 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:54.045764 2556 generic.go:356] "Generic (PLEG): container finished" podID="81ef2a97-c51e-4d07-b26d-083e4e348612" containerID="5191e029bf6ac61a25172102b677e01e18a61f0f23d57550cd62535f6440bca2" exitCode=255 Jul 02 16:58:54.045816 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:54.045791 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" event={"ID":"81ef2a97-c51e-4d07-b26d-083e4e348612","Type":"ContainerDied","Data":"5191e029bf6ac61a25172102b677e01e18a61f0f23d57550cd62535f6440bca2"} Jul 02 16:58:54.045917 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:54.045834 2556 scope.go:117] "RemoveContainer" containerID="52202971f34e8e3d358cc4116d147a1f54c1a1b2a27096e2a8a443eae21ef6fc" Jul 02 16:58:54.046192 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:54.046171 2556 scope.go:117] "RemoveContainer" containerID="5191e029bf6ac61a25172102b677e01e18a61f0f23d57550cd62535f6440bca2" Jul 02 16:58:54.046395 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:54.046367 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-njvpd_openshift-console-operator(81ef2a97-c51e-4d07-b26d-083e4e348612)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" podUID="81ef2a97-c51e-4d07-b26d-083e4e348612" Jul 02 16:58:55.050018 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:55.049992 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 16:58:56.884879 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:56.884842 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c9f9f68b6-xmx27"] Jul 02 16:58:56.896645 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:56.896621 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:56.900844 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:56.900631 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 02 16:58:56.900844 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:56.900693 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s9rkj\"" Jul 02 16:58:56.900844 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:56.900715 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c9f9f68b6-xmx27"] Jul 02 16:58:56.901078 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:56.900877 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 02 16:58:56.901078 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:56.901000 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 02 16:58:56.904873 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:56.904856 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 02 16:58:57.063456 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.063433 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90c0aef6-7f79-4c5e-96e5-811086bcad5e-trusted-ca\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.063607 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.063495 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90c0aef6-7f79-4c5e-96e5-811086bcad5e-image-registry-private-configuration\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.063607 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.063536 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90c0aef6-7f79-4c5e-96e5-811086bcad5e-registry-tls\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.063607 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.063573 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90c0aef6-7f79-4c5e-96e5-811086bcad5e-registry-certificates\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.063773 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.063611 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90c0aef6-7f79-4c5e-96e5-811086bcad5e-ca-trust-extracted\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.063773 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.063641 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90c0aef6-7f79-4c5e-96e5-811086bcad5e-bound-sa-token\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.063773 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.063693 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kl6w\" (UniqueName: \"kubernetes.io/projected/90c0aef6-7f79-4c5e-96e5-811086bcad5e-kube-api-access-5kl6w\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.063773 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.063765 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90c0aef6-7f79-4c5e-96e5-811086bcad5e-installation-pull-secrets\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.164738 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.164668 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90c0aef6-7f79-4c5e-96e5-811086bcad5e-image-registry-private-configuration\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.164738 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.164697 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90c0aef6-7f79-4c5e-96e5-811086bcad5e-registry-tls\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.164738 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.164719 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90c0aef6-7f79-4c5e-96e5-811086bcad5e-registry-certificates\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.164954 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.164741 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90c0aef6-7f79-4c5e-96e5-811086bcad5e-ca-trust-extracted\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.164954 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.164794 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90c0aef6-7f79-4c5e-96e5-811086bcad5e-bound-sa-token\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.164954 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.164830 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kl6w\" (UniqueName: \"kubernetes.io/projected/90c0aef6-7f79-4c5e-96e5-811086bcad5e-kube-api-access-5kl6w\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.164954 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.164890 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90c0aef6-7f79-4c5e-96e5-811086bcad5e-installation-pull-secrets\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.164954 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.164925 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90c0aef6-7f79-4c5e-96e5-811086bcad5e-trusted-ca\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.165365 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.165340 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90c0aef6-7f79-4c5e-96e5-811086bcad5e-ca-trust-extracted\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.165729 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.165711 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90c0aef6-7f79-4c5e-96e5-811086bcad5e-registry-certificates\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.166447 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.166423 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90c0aef6-7f79-4c5e-96e5-811086bcad5e-trusted-ca\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.167308 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.167275 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90c0aef6-7f79-4c5e-96e5-811086bcad5e-image-registry-private-configuration\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.167536 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.167514 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90c0aef6-7f79-4c5e-96e5-811086bcad5e-registry-tls\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.167536 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.167528 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90c0aef6-7f79-4c5e-96e5-811086bcad5e-installation-pull-secrets\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.171780 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.171753 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90c0aef6-7f79-4c5e-96e5-811086bcad5e-bound-sa-token\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.172589 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.172563 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kl6w\" (UniqueName: \"kubernetes.io/projected/90c0aef6-7f79-4c5e-96e5-811086bcad5e-kube-api-access-5kl6w\") pod \"image-registry-c9f9f68b6-xmx27\" (UID: \"90c0aef6-7f79-4c5e-96e5-811086bcad5e\") " pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.207095 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.207078 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:57.351268 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:57.351241 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c9f9f68b6-xmx27"] Jul 02 16:58:57.354236 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:58:57.354213 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90c0aef6_7f79_4c5e_96e5_811086bcad5e.slice/crio-6aae496f74f725996689ac2ec507167e8957d2866918a122a109dd527f76a974 WatchSource:0}: Error finding container 6aae496f74f725996689ac2ec507167e8957d2866918a122a109dd527f76a974: Status 404 returned error can't find the container with id 6aae496f74f725996689ac2ec507167e8957d2866918a122a109dd527f76a974 Jul 02 16:58:58.061013 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:58.060972 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" event={"ID":"90c0aef6-7f79-4c5e-96e5-811086bcad5e","Type":"ContainerStarted","Data":"54b3188e7b634a44aaff7bb095fb4ef39dc11a1a4b7bf0b38000620ba7811043"} Jul 02 16:58:58.061381 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:58.061067 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" event={"ID":"90c0aef6-7f79-4c5e-96e5-811086bcad5e","Type":"ContainerStarted","Data":"6aae496f74f725996689ac2ec507167e8957d2866918a122a109dd527f76a974"} Jul 02 16:58:58.061381 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:58.061113 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:58:58.079105 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:58.079060 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" podStartSLOduration=2.079048309 podStartE2EDuration="2.079048309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:58:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:58:58.07749371 +0000 UTC m=+271.390367355" watchObservedRunningTime="2026-07-02 16:58:58.079048309 +0000 UTC m=+271.391921955" Jul 02 16:58:58.854628 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:58.854598 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:58.854788 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:58.854650 2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:58:58.854894 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:58:58.854883 2556 scope.go:117] "RemoveContainer" containerID="5191e029bf6ac61a25172102b677e01e18a61f0f23d57550cd62535f6440bca2" Jul 02 16:58:58.855045 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:58:58.855028 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-njvpd_openshift-console-operator(81ef2a97-c51e-4d07-b26d-083e4e348612)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" podUID="81ef2a97-c51e-4d07-b26d-083e4e348612" Jul 02 16:59:02.913639 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:02.913605 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-t8hhj"] Jul 02 16:59:02.916475 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:02.916459 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-t8hhj" Jul 02 16:59:02.918754 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:02.918730 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 02 16:59:02.919416 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:02.919389 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-ngd82\"" Jul 02 16:59:02.926024 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:02.926004 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-t8hhj"] Jul 02 16:59:03.110101 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:03.110070 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bd1e6dfc-555f-4f6e-95d2-bae33e28e7c1-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-t8hhj\" (UID: \"bd1e6dfc-555f-4f6e-95d2-bae33e28e7c1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-t8hhj" Jul 02 16:59:03.211024 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:03.210959 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bd1e6dfc-555f-4f6e-95d2-bae33e28e7c1-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-t8hhj\" (UID: \"bd1e6dfc-555f-4f6e-95d2-bae33e28e7c1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-t8hhj" Jul 02 16:59:03.213273 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:03.213253 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bd1e6dfc-555f-4f6e-95d2-bae33e28e7c1-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-t8hhj\" (UID: \"bd1e6dfc-555f-4f6e-95d2-bae33e28e7c1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-t8hhj" Jul 02 16:59:03.227266 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:03.227241 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-t8hhj" Jul 02 16:59:03.343561 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:03.343540 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-t8hhj"] Jul 02 16:59:03.345911 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:59:03.345878 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd1e6dfc_555f_4f6e_95d2_bae33e28e7c1.slice/crio-9006cd31ef1c2d3e3adccdc023ef2eea0e70024b4dcb51fe7be4d8a93ae94350 WatchSource:0}: Error finding container 9006cd31ef1c2d3e3adccdc023ef2eea0e70024b4dcb51fe7be4d8a93ae94350: Status 404 returned error can't find the container with id 9006cd31ef1c2d3e3adccdc023ef2eea0e70024b4dcb51fe7be4d8a93ae94350 Jul 02 16:59:04.080871 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:04.080838 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-t8hhj" event={"ID":"bd1e6dfc-555f-4f6e-95d2-bae33e28e7c1","Type":"ContainerStarted","Data":"9006cd31ef1c2d3e3adccdc023ef2eea0e70024b4dcb51fe7be4d8a93ae94350"} Jul 02 16:59:05.085990 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:05.085953 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-t8hhj" event={"ID":"bd1e6dfc-555f-4f6e-95d2-bae33e28e7c1","Type":"ContainerStarted","Data":"bd4f9652b98f450b5383c1145c3d3d2ed71918880d915326a9d805743eeda534"} Jul 02 16:59:05.086416 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:05.086130 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-t8hhj" Jul 02 16:59:05.090683 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:05.090655 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-t8hhj" Jul 02 16:59:05.100022 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:05.099984 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-t8hhj" podStartSLOduration=2.052193729 podStartE2EDuration="3.099970761s" totalImagesPullingTime="1.047777032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:59:05.099539287 +0000 UTC m=+278.412412932" watchObservedRunningTime="2026-07-02 16:59:05.099970761 +0000 UTC m=+278.412844408" Jul 02 16:59:05.530836 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:05.530809 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:59:05.533191 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:05.533149 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82bb767c-600e-4125-bf49-254356619b37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4l2sg\" (UID: \"82bb767c-600e-4125-bf49-254356619b37\") " pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:59:05.792224 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:05.792117 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4l2sg" Jul 02 16:59:05.913631 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:05.913604 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4l2sg"] Jul 02 16:59:05.916678 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:59:05.916654 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82bb767c_600e_4125_bf49_254356619b37.slice/crio-158764c60d8e968541eb04de54d369b2a851e41fe145ce39da6fad193edc541c WatchSource:0}: Error finding container 158764c60d8e968541eb04de54d369b2a851e41fe145ce39da6fad193edc541c: Status 404 returned error can't find the container with id 158764c60d8e968541eb04de54d369b2a851e41fe145ce39da6fad193edc541c Jul 02 16:59:06.089889 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:06.089821 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4l2sg" event={"ID":"82bb767c-600e-4125-bf49-254356619b37","Type":"ContainerStarted","Data":"158764c60d8e968541eb04de54d369b2a851e41fe145ce39da6fad193edc541c"} Jul 02 16:59:08.732179 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:59:08.732124 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-76lqz" podUID="eca22f13-1929-48b6-90fd-5d82f58f6017" Jul 02 16:59:09.101295 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:09.101267 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-76lqz" Jul 02 16:59:10.105926 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:10.105884 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4l2sg" event={"ID":"82bb767c-600e-4125-bf49-254356619b37","Type":"ContainerStarted","Data":"1f594db4af3f2505a6ea08eb94ae84f151773368c0486a8e3baa84f320200dab"} Jul 02 16:59:11.111552 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:11.111511 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4l2sg" event={"ID":"82bb767c-600e-4125-bf49-254356619b37","Type":"ContainerStarted","Data":"247d6f9cc7b334de64b6362f4bbdb89263b1596e405fa7ae3de195179a8fcd6f"} Jul 02 16:59:11.184630 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:11.184594 2556 scope.go:117] "RemoveContainer" containerID="5191e029bf6ac61a25172102b677e01e18a61f0f23d57550cd62535f6440bca2" Jul 02 16:59:11.184955 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:59:11.184790 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-njvpd_openshift-console-operator(81ef2a97-c51e-4d07-b26d-083e4e348612)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" podUID="81ef2a97-c51e-4d07-b26d-083e4e348612" Jul 02 16:59:11.983117 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:11.983083 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:59:11.985709 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:11.985680 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eca22f13-1929-48b6-90fd-5d82f58f6017-metrics-tls\") pod \"dns-default-76lqz\" (UID: \"eca22f13-1929-48b6-90fd-5d82f58f6017\") " pod="openshift-dns/dns-default-76lqz" Jul 02 16:59:12.105092 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:12.105059 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mmn65\"" Jul 02 16:59:12.112252 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:12.112226 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-76lqz" Jul 02 16:59:12.963202 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:12.963112 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-76lqz"] Jul 02 16:59:12.965433 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:59:12.965410 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca22f13_1929_48b6_90fd_5d82f58f6017.slice/crio-38d5eebcc89d89871274ca6941d2991b0478ac5cbbec4a5f94a79a8e8cce3e74 WatchSource:0}: Error finding container 38d5eebcc89d89871274ca6941d2991b0478ac5cbbec4a5f94a79a8e8cce3e74: Status 404 returned error can't find the container with id 38d5eebcc89d89871274ca6941d2991b0478ac5cbbec4a5f94a79a8e8cce3e74 Jul 02 16:59:13.121169 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:13.121080 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4l2sg" event={"ID":"82bb767c-600e-4125-bf49-254356619b37","Type":"ContainerStarted","Data":"52cec89cf40a49a27e08ddaa13ad718eaa5ce3fb7d2160b40d5969732b903005"} Jul 02 16:59:13.122255 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:13.122237 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-76lqz" event={"ID":"eca22f13-1929-48b6-90fd-5d82f58f6017","Type":"ContainerStarted","Data":"38d5eebcc89d89871274ca6941d2991b0478ac5cbbec4a5f94a79a8e8cce3e74"} Jul 02 16:59:13.138758 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:13.138714 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4l2sg" podStartSLOduration=33.274469838 podStartE2EDuration="40.138699759s" totalImagesPullingTime="6.864229921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:58:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:59:13.138110999 +0000 UTC m=+286.450984655" watchObservedRunningTime="2026-07-02 16:59:13.138699759 +0000 UTC m=+286.451573405" Jul 02 16:59:15.131910 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:15.131876 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-76lqz" event={"ID":"eca22f13-1929-48b6-90fd-5d82f58f6017","Type":"ContainerStarted","Data":"2b637fa1c375473ce5fd033c7fbdd6409d560cb56617ab85949d9a66eaf41675"} Jul 02 16:59:15.132304 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:15.131953 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-76lqz" event={"ID":"eca22f13-1929-48b6-90fd-5d82f58f6017","Type":"ContainerStarted","Data":"e3b9ea0fc75f62fdc9b878e6ddbf15a865df017c4bb5f026dc3a5a37cf219893"} Jul 02 16:59:15.132304 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:15.131974 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-76lqz" Jul 02 16:59:15.148390 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:15.148345 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-76lqz" podStartSLOduration=252.618917497 podStartE2EDuration="4m14.148330725s" totalImagesPullingTime="1.529413228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:55:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:59:15.14683163 +0000 UTC m=+288.459705276" watchObservedRunningTime="2026-07-02 16:59:15.148330725 +0000 UTC m=+288.461204370" Jul 02 16:59:17.211393 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:17.211359 2556 patch_prober.go:28] interesting pod/image-registry-c9f9f68b6-xmx27 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 02 16:59:17.211762 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:17.211405 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" podUID="90c0aef6-7f79-4c5e-96e5-811086bcad5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 16:59:18.275588 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.275557 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-smszq"] Jul 02 16:59:18.278972 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.278954 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.282351 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.282320 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 16:59:18.282825 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.282806 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 16:59:18.283086 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.283070 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 16:59:18.283192 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.283101 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 16:59:18.283192 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.283102 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4v478\"" Jul 02 16:59:18.291513 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.291497 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 16:59:18.292035 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.292003 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 16:59:18.335491 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.335472 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.335606 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.335504 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-accelerators-collector-config\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.335606 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.335545 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-textfile\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.335606 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.335568 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-tls\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.335606 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.335593 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4582a064-0ea5-4c77-9e29-ed41109a0325-metrics-client-ca\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.335742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.335668 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4582a064-0ea5-4c77-9e29-ed41109a0325-root\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.335742 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.335723 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-wtmp\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.335843 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.335761 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmwpd\" (UniqueName: \"kubernetes.io/projected/4582a064-0ea5-4c77-9e29-ed41109a0325-kube-api-access-zmwpd\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.335843 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.335779 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4582a064-0ea5-4c77-9e29-ed41109a0325-sys\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.436507 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.436483 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-textfile\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.436589 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.436510 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-tls\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.436589 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.436528 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4582a064-0ea5-4c77-9e29-ed41109a0325-metrics-client-ca\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.436589 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.436560 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4582a064-0ea5-4c77-9e29-ed41109a0325-root\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.436589 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.436577 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-wtmp\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.436796 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.436612 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmwpd\" (UniqueName: \"kubernetes.io/projected/4582a064-0ea5-4c77-9e29-ed41109a0325-kube-api-access-zmwpd\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.436796 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.436628 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4582a064-0ea5-4c77-9e29-ed41109a0325-sys\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.436796 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:59:18.436632 2556 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 02 16:59:18.436796 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.436640 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4582a064-0ea5-4c77-9e29-ed41109a0325-root\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.436796 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.436649 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.436796 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:59:18.436682 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-tls podName:4582a064-0ea5-4c77-9e29-ed41109a0325 nodeName:}" failed. No retries permitted until 2026-07-02 16:59:18.936666044 +0000 UTC m=+292.249539668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-tls") pod "node-exporter-smszq" (UID: "4582a064-0ea5-4c77-9e29-ed41109a0325") : secret "node-exporter-tls" not found Jul 02 16:59:18.436796 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.436731 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4582a064-0ea5-4c77-9e29-ed41109a0325-sys\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.436796 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.436781 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-accelerators-collector-config\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.436796 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.436789 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-wtmp\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.437261 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.436838 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-textfile\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.437261 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.437193 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4582a064-0ea5-4c77-9e29-ed41109a0325-metrics-client-ca\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.437261 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.437223 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-accelerators-collector-config\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.439231 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.439211 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.444671 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.444653 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmwpd\" (UniqueName: \"kubernetes.io/projected/4582a064-0ea5-4c77-9e29-ed41109a0325-kube-api-access-zmwpd\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.941333 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.941294 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-tls\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:18.943582 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:18.943559 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4582a064-0ea5-4c77-9e29-ed41109a0325-node-exporter-tls\") pod \"node-exporter-smszq\" (UID: \"4582a064-0ea5-4c77-9e29-ed41109a0325\") " pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:19.068740 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:19.068714 2556 patch_prober.go:28] interesting pod/image-registry-c9f9f68b6-xmx27 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 02 16:59:19.068885 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:19.068763 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" podUID="90c0aef6-7f79-4c5e-96e5-811086bcad5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 16:59:19.188444 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:19.188416 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-smszq" Jul 02 16:59:19.198377 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:59:19.198356 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4582a064_0ea5_4c77_9e29_ed41109a0325.slice/crio-df31ae14d6b020e25f9a1efb389831661c6b5ac3f2277027e5ef67a87f1556cf WatchSource:0}: Error finding container df31ae14d6b020e25f9a1efb389831661c6b5ac3f2277027e5ef67a87f1556cf: Status 404 returned error can't find the container with id df31ae14d6b020e25f9a1efb389831661c6b5ac3f2277027e5ef67a87f1556cf Jul 02 16:59:20.150280 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.150245 2556 generic.go:356] "Generic (PLEG): container finished" podID="4582a064-0ea5-4c77-9e29-ed41109a0325" containerID="f67895a299b908005a09bbf4374dc343b44c635d042b587edae3f08ff0d9e1b8" exitCode=0 Jul 02 16:59:20.150590 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.150320 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-smszq" event={"ID":"4582a064-0ea5-4c77-9e29-ed41109a0325","Type":"ContainerDied","Data":"f67895a299b908005a09bbf4374dc343b44c635d042b587edae3f08ff0d9e1b8"} Jul 02 16:59:20.150590 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.150372 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-smszq" event={"ID":"4582a064-0ea5-4c77-9e29-ed41109a0325","Type":"ContainerStarted","Data":"df31ae14d6b020e25f9a1efb389831661c6b5ac3f2277027e5ef67a87f1556cf"} Jul 02 16:59:20.211889 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.211863 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5d45fb65c-wtv7z"] Jul 02 16:59:20.216628 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.216614 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.219364 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.219340 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 02 16:59:20.219469 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.219447 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-47ov2otb1ub8n\"" Jul 02 16:59:20.219536 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.219478 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 02 16:59:20.219536 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.219480 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 02 16:59:20.219646 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.219480 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 02 16:59:20.219646 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.219600 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-g6wv2\"" Jul 02 16:59:20.219827 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.219811 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 02 16:59:20.228142 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.228120 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d45fb65c-wtv7z"] Jul 02 16:59:20.252148 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.252124 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-grpc-tls\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.252254 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.252193 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.252254 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.252224 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-thanos-querier-tls\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.252324 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.252303 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.252410 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.252391 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.252466 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.252439 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg2g5\" (UniqueName: \"kubernetes.io/projected/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-kube-api-access-gg2g5\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.252600 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.252581 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.252658 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.252645 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-metrics-client-ca\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.353447 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.353418 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.353584 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.353460 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-thanos-querier-tls\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.353584 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.353504 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.353689 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.353641 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.353689 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.353683 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg2g5\" (UniqueName: \"kubernetes.io/projected/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-kube-api-access-gg2g5\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.353790 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.353721 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.353790 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.353753 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-metrics-client-ca\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.353901 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.353800 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-grpc-tls\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.355088 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.355053 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-metrics-client-ca\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.356102 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.356079 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.356426 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.356404 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.356477 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.356445 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.356477 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.356463 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-thanos-querier-tls\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.356545 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.356487 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-grpc-tls\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.356728 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.356711 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.361440 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.361419 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg2g5\" (UniqueName: \"kubernetes.io/projected/ad94a6a3-4de7-472c-b756-fb09c79dcbd5-kube-api-access-gg2g5\") pod \"thanos-querier-5d45fb65c-wtv7z\" (UID: \"ad94a6a3-4de7-472c-b756-fb09c79dcbd5\") " pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.525393 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.525374 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:20.661922 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:20.661891 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d45fb65c-wtv7z"] Jul 02 16:59:20.664566 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:59:20.664544 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad94a6a3_4de7_472c_b756_fb09c79dcbd5.slice/crio-c788bf6d6494b70453eb9a63850013ba88545ab723abc4f3d02136a51d4b8c09 WatchSource:0}: Error finding container c788bf6d6494b70453eb9a63850013ba88545ab723abc4f3d02136a51d4b8c09: Status 404 returned error can't find the container with id c788bf6d6494b70453eb9a63850013ba88545ab723abc4f3d02136a51d4b8c09 Jul 02 16:59:21.156174 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:21.156122 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-smszq" event={"ID":"4582a064-0ea5-4c77-9e29-ed41109a0325","Type":"ContainerStarted","Data":"8ca857cf632d54e57311ddffe752db60d7a307eea4b015c95a221be503e1721b"} Jul 02 16:59:21.156602 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:21.156197 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-smszq" event={"ID":"4582a064-0ea5-4c77-9e29-ed41109a0325","Type":"ContainerStarted","Data":"56d8354ea72c0efd3f2f6528872c5254b80106565b2c589593c204b257c67ba8"} Jul 02 16:59:21.157378 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:21.157359 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" event={"ID":"ad94a6a3-4de7-472c-b756-fb09c79dcbd5","Type":"ContainerStarted","Data":"c788bf6d6494b70453eb9a63850013ba88545ab723abc4f3d02136a51d4b8c09"} Jul 02 16:59:21.176647 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:21.176606 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-smszq" podStartSLOduration=2.332906044 podStartE2EDuration="3.176593258s" totalImagesPullingTime="781.060519ms" totalInitContainerRuntime="62.626695ms" isStatefulPod=true podCreationTimestamp="2026-07-02 16:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:59:21.175005525 +0000 UTC m=+294.487879171" watchObservedRunningTime="2026-07-02 16:59:21.176593258 +0000 UTC m=+294.489466905" Jul 02 16:59:22.184452 ip-10-0-129-102 kubenswrapper[2556]: E0702 16:59:22.184411 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-bzrql" podUID="6c35723b-8e1e-4682-b963-70e3266a6a71" Jul 02 16:59:23.166711 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:23.166674 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" event={"ID":"ad94a6a3-4de7-472c-b756-fb09c79dcbd5","Type":"ContainerStarted","Data":"686267219e55e45ffb6bc73cd3f6bb7d2fc1dd29c58bdeb2cdaa3c84e55bfc1a"} Jul 02 16:59:23.166841 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:23.166764 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" event={"ID":"ad94a6a3-4de7-472c-b756-fb09c79dcbd5","Type":"ContainerStarted","Data":"cdc59e9ab4efa443a61ce454fa28797e55024196025f65636e593b9434fbd113"} Jul 02 16:59:23.166841 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:23.166791 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" event={"ID":"ad94a6a3-4de7-472c-b756-fb09c79dcbd5","Type":"ContainerStarted","Data":"0fa40e705519f4184a440e66812d7b2593bcc93a71b0f8d72d17f1b51cfe8b08"} Jul 02 16:59:24.173559 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:24.173467 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" event={"ID":"ad94a6a3-4de7-472c-b756-fb09c79dcbd5","Type":"ContainerStarted","Data":"45a659a5ca0981b0241ff6860a3db04eb5e70b155bd4c2214d721b8ed5b97eb9"} Jul 02 16:59:24.173559 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:24.173534 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" event={"ID":"ad94a6a3-4de7-472c-b756-fb09c79dcbd5","Type":"ContainerStarted","Data":"da0380ef4e5430e57b38b6cf70bc1cdb16de2264d103c8e2103d4aa5510466a9"} Jul 02 16:59:24.173559 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:24.173561 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" event={"ID":"ad94a6a3-4de7-472c-b756-fb09c79dcbd5","Type":"ContainerStarted","Data":"8fd4ecd867ee687dd7e65803aac4d76d5e9307846d64c7042baae1b3bc48bba3"} Jul 02 16:59:24.174104 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:24.173644 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:24.199243 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:24.199197 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" podStartSLOduration=1.132121579 podStartE2EDuration="4.199181338s" totalImagesPullingTime="3.067059759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:59:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:59:24.197447187 +0000 UTC m=+297.510320824" watchObservedRunningTime="2026-07-02 16:59:24.199181338 +0000 UTC m=+297.512054978" Jul 02 16:59:25.136939 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:25.136912 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-76lqz" Jul 02 16:59:25.184304 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:25.184280 2556 scope.go:117] "RemoveContainer" containerID="5191e029bf6ac61a25172102b677e01e18a61f0f23d57550cd62535f6440bca2" Jul 02 16:59:25.771822 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:25.771792 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5499b6fb8f-fwk25"] Jul 02 16:59:25.774916 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:25.774897 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-fwk25" Jul 02 16:59:25.777586 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:25.777565 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 16:59:25.777661 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:25.777588 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 16:59:25.777661 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:25.777575 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-vf2gb\"" Jul 02 16:59:25.789018 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:25.786716 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-fwk25"] Jul 02 16:59:25.902046 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:25.902018 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gprsj\" (UniqueName: \"kubernetes.io/projected/0ee604fa-e2f0-4616-978a-ef689e30fd4a-kube-api-access-gprsj\") pod \"downloads-5499b6fb8f-fwk25\" (UID: \"0ee604fa-e2f0-4616-978a-ef689e30fd4a\") " pod="openshift-console/downloads-5499b6fb8f-fwk25" Jul 02 16:59:26.002863 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:26.002839 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gprsj\" (UniqueName: \"kubernetes.io/projected/0ee604fa-e2f0-4616-978a-ef689e30fd4a-kube-api-access-gprsj\") pod \"downloads-5499b6fb8f-fwk25\" (UID: \"0ee604fa-e2f0-4616-978a-ef689e30fd4a\") " pod="openshift-console/downloads-5499b6fb8f-fwk25" Jul 02 16:59:26.014237 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:26.014212 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gprsj\" (UniqueName: \"kubernetes.io/projected/0ee604fa-e2f0-4616-978a-ef689e30fd4a-kube-api-access-gprsj\") pod \"downloads-5499b6fb8f-fwk25\" (UID: \"0ee604fa-e2f0-4616-978a-ef689e30fd4a\") " pod="openshift-console/downloads-5499b6fb8f-fwk25" Jul 02 16:59:26.084319 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:26.084268 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-fwk25" Jul 02 16:59:26.183929 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:26.183905 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 16:59:26.184061 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:26.183962 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" event={"ID":"81ef2a97-c51e-4d07-b26d-083e4e348612","Type":"ContainerStarted","Data":"39adc21719e8e71c275ae750e5d665d5e393ffb79b8928c6b591beebf07ce580"} Jul 02 16:59:26.184294 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:26.184275 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:59:26.189352 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:26.189329 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" Jul 02 16:59:26.202354 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:26.202315 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7fdbdf9c99-njvpd" podStartSLOduration=54.847002891 podStartE2EDuration="58.202300166s" totalImagesPullingTime="3.355297275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:59:26.201871051 +0000 UTC m=+299.514744698" watchObservedRunningTime="2026-07-02 16:59:26.202300166 +0000 UTC m=+299.515173812" Jul 02 16:59:26.207796 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:26.207774 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-fwk25"] Jul 02 16:59:26.211640 ip-10-0-129-102 kubenswrapper[2556]: W0702 16:59:26.211564 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ee604fa_e2f0_4616_978a_ef689e30fd4a.slice/crio-26f7873e5ae9d999658d356a43cc996b41cc1b573fe16a053f4d30645c3cd32c WatchSource:0}: Error finding container 26f7873e5ae9d999658d356a43cc996b41cc1b573fe16a053f4d30645c3cd32c: Status 404 returned error can't find the container with id 26f7873e5ae9d999658d356a43cc996b41cc1b573fe16a053f4d30645c3cd32c Jul 02 16:59:27.112752 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:27.112724 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 16:59:27.113284 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:27.113252 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 16:59:27.125561 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:27.125535 2556 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 16:59:27.189677 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:27.189644 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-fwk25" event={"ID":"0ee604fa-e2f0-4616-978a-ef689e30fd4a","Type":"ContainerStarted","Data":"26f7873e5ae9d999658d356a43cc996b41cc1b573fe16a053f4d30645c3cd32c"} Jul 02 16:59:27.212112 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:27.211967 2556 patch_prober.go:28] interesting pod/image-registry-c9f9f68b6-xmx27 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 02 16:59:27.212112 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:27.212024 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" podUID="90c0aef6-7f79-4c5e-96e5-811086bcad5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 16:59:29.068887 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:29.068820 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c9f9f68b6-xmx27" Jul 02 16:59:30.185529 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:30.185499 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5d45fb65c-wtv7z" Jul 02 16:59:37.185258 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:37.185226 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 16:59:45.272358 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:45.272313 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-fwk25" event={"ID":"0ee604fa-e2f0-4616-978a-ef689e30fd4a","Type":"ContainerStarted","Data":"7073863efd01b46e76c0da81d23a3e140fa8248ca85b5a080ecd5790f31a6d6e"} Jul 02 16:59:45.272850 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:45.272545 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5499b6fb8f-fwk25" Jul 02 16:59:45.280296 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:45.280266 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5499b6fb8f-fwk25" Jul 02 16:59:45.289962 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:45.289908 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5499b6fb8f-fwk25" podStartSLOduration=2.066920222 podStartE2EDuration="20.289894245s" totalImagesPullingTime="18.222974023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:59:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:59:45.288473254 +0000 UTC m=+318.601346902" watchObservedRunningTime="2026-07-02 16:59:45.289894245 +0000 UTC m=+318.602767893" Jul 02 16:59:48.286709 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:48.286675 2556 generic.go:356] "Generic (PLEG): container finished" podID="784b2438-38ab-4c11-a265-ee0b046adfb0" containerID="9c22dab5577ab80c1747493eeb6fa96b2196a5a6166dfc0332225f80b01cd072" exitCode=0 Jul 02 16:59:48.287216 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:48.286756 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm" event={"ID":"784b2438-38ab-4c11-a265-ee0b046adfb0","Type":"ContainerDied","Data":"9c22dab5577ab80c1747493eeb6fa96b2196a5a6166dfc0332225f80b01cd072"} Jul 02 16:59:48.287216 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:48.287188 2556 scope.go:117] "RemoveContainer" containerID="9c22dab5577ab80c1747493eeb6fa96b2196a5a6166dfc0332225f80b01cd072" Jul 02 16:59:49.293608 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:49.293567 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-ghvnm" event={"ID":"784b2438-38ab-4c11-a265-ee0b046adfb0","Type":"ContainerStarted","Data":"7ac56744112477fd5784931b614c15a00caa4b3343fffb0353b28d87147a3f72"} Jul 02 16:59:53.313244 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:53.313208 2556 generic.go:356] "Generic (PLEG): container finished" podID="4d1a424d-f615-4619-9406-42979581d953" containerID="0808bc93f81236a42d18a49b3a81df86a552b69ccbcfe2a8517de6e57ec77b9c" exitCode=0 Jul 02 16:59:53.313661 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:53.313295 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-gzntn" event={"ID":"4d1a424d-f615-4619-9406-42979581d953","Type":"ContainerDied","Data":"0808bc93f81236a42d18a49b3a81df86a552b69ccbcfe2a8517de6e57ec77b9c"} Jul 02 16:59:53.313753 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:53.313705 2556 scope.go:117] "RemoveContainer" containerID="0808bc93f81236a42d18a49b3a81df86a552b69ccbcfe2a8517de6e57ec77b9c" Jul 02 16:59:54.318784 ip-10-0-129-102 kubenswrapper[2556]: I0702 16:59:54.318751 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-gzntn" event={"ID":"4d1a424d-f615-4619-9406-42979581d953","Type":"ContainerStarted","Data":"56066d6741cf15a26bcf51b62e03fa223a96c4dab3b998839084e3e8ee7a6509"} Jul 02 17:00:41.109822 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:00:41.109787 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs\") pod \"network-metrics-daemon-bzrql\" (UID: \"6c35723b-8e1e-4682-b963-70e3266a6a71\") " pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 17:00:41.112125 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:00:41.112103 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c35723b-8e1e-4682-b963-70e3266a6a71-metrics-certs\") pod \"network-metrics-daemon-bzrql\" (UID: \"6c35723b-8e1e-4682-b963-70e3266a6a71\") " pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 17:00:41.388872 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:00:41.388800 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6k5zx\"" Jul 02 17:00:41.396178 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:00:41.396153 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bzrql" Jul 02 17:00:41.520066 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:00:41.520040 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bzrql"] Jul 02 17:00:41.522332 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:00:41.522307 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c35723b_8e1e_4682_b963_70e3266a6a71.slice/crio-2c36e0deae930d6ec841a9f9578eaf7032466d20dcb7157486e011cfe2349fe6 WatchSource:0}: Error finding container 2c36e0deae930d6ec841a9f9578eaf7032466d20dcb7157486e011cfe2349fe6: Status 404 returned error can't find the container with id 2c36e0deae930d6ec841a9f9578eaf7032466d20dcb7157486e011cfe2349fe6 Jul 02 17:00:41.524175 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:00:41.524144 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 17:00:42.503065 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:00:42.503032 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bzrql" event={"ID":"6c35723b-8e1e-4682-b963-70e3266a6a71","Type":"ContainerStarted","Data":"72f7ea69c27b7d53425a1c276a05c962105e176595492da55a98184bf0e69be8"} Jul 02 17:00:42.503465 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:00:42.503099 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bzrql" event={"ID":"6c35723b-8e1e-4682-b963-70e3266a6a71","Type":"ContainerStarted","Data":"2c36e0deae930d6ec841a9f9578eaf7032466d20dcb7157486e011cfe2349fe6"} Jul 02 17:00:43.509199 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:00:43.509139 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bzrql" event={"ID":"6c35723b-8e1e-4682-b963-70e3266a6a71","Type":"ContainerStarted","Data":"69b77128a3a835ce44a8468d58ace3c7f91b57747a64227a45887b1c4568f5dd"} Jul 02 17:00:43.528875 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:00:43.528828 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bzrql" podStartSLOduration=375.642001068 podStartE2EDuration="6m16.528814084s" totalImagesPullingTime="886.813016ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:00:43.526254599 +0000 UTC m=+376.839128246" watchObservedRunningTime="2026-07-02 17:00:43.528814084 +0000 UTC m=+376.841687729" Jul 02 17:02:23.320135 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:23.320060 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-p8xkg"] Jul 02 17:02:23.322075 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:23.322059 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p8xkg" Jul 02 17:02:23.324544 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:23.324519 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 02 17:02:23.325573 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:23.325553 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8mkpr\"" Jul 02 17:02:23.325673 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:23.325578 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 02 17:02:23.332704 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:23.332684 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-p8xkg"] Jul 02 17:02:23.447341 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:23.447312 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2fzh\" (UniqueName: \"kubernetes.io/projected/5c88a6cb-f4d0-4a92-a833-5c9260b1670b-kube-api-access-l2fzh\") pod \"cert-manager-cainjector-559cdd8c87-p8xkg\" (UID: \"5c88a6cb-f4d0-4a92-a833-5c9260b1670b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p8xkg" Jul 02 17:02:23.447341 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:23.447350 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c88a6cb-f4d0-4a92-a833-5c9260b1670b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-p8xkg\" (UID: \"5c88a6cb-f4d0-4a92-a833-5c9260b1670b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p8xkg" Jul 02 17:02:23.548053 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:23.548020 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2fzh\" (UniqueName: \"kubernetes.io/projected/5c88a6cb-f4d0-4a92-a833-5c9260b1670b-kube-api-access-l2fzh\") pod \"cert-manager-cainjector-559cdd8c87-p8xkg\" (UID: \"5c88a6cb-f4d0-4a92-a833-5c9260b1670b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p8xkg" Jul 02 17:02:23.548225 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:23.548064 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c88a6cb-f4d0-4a92-a833-5c9260b1670b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-p8xkg\" (UID: \"5c88a6cb-f4d0-4a92-a833-5c9260b1670b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p8xkg" Jul 02 17:02:23.556401 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:23.556368 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c88a6cb-f4d0-4a92-a833-5c9260b1670b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-p8xkg\" (UID: \"5c88a6cb-f4d0-4a92-a833-5c9260b1670b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p8xkg" Jul 02 17:02:23.556504 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:23.556473 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2fzh\" (UniqueName: \"kubernetes.io/projected/5c88a6cb-f4d0-4a92-a833-5c9260b1670b-kube-api-access-l2fzh\") pod \"cert-manager-cainjector-559cdd8c87-p8xkg\" (UID: \"5c88a6cb-f4d0-4a92-a833-5c9260b1670b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p8xkg" Jul 02 17:02:23.630536 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:23.630455 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p8xkg" Jul 02 17:02:23.751523 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:23.751495 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-p8xkg"] Jul 02 17:02:23.754633 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:02:23.754613 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c88a6cb_f4d0_4a92_a833_5c9260b1670b.slice/crio-89eb78b24c27f22aef147eca10b1c46c973eff56f51af7d941e08abe96ef4cba WatchSource:0}: Error finding container 89eb78b24c27f22aef147eca10b1c46c973eff56f51af7d941e08abe96ef4cba: Status 404 returned error can't find the container with id 89eb78b24c27f22aef147eca10b1c46c973eff56f51af7d941e08abe96ef4cba Jul 02 17:02:23.884882 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:23.884801 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p8xkg" event={"ID":"5c88a6cb-f4d0-4a92-a833-5c9260b1670b","Type":"ContainerStarted","Data":"89eb78b24c27f22aef147eca10b1c46c973eff56f51af7d941e08abe96ef4cba"} Jul 02 17:02:26.899693 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:26.899657 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p8xkg" event={"ID":"5c88a6cb-f4d0-4a92-a833-5c9260b1670b","Type":"ContainerStarted","Data":"41ac299bc2159b024a920be3affd5baeac519ef618d58040f8a2aeb230a61436"} Jul 02 17:02:26.914421 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:26.914338 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p8xkg" podStartSLOduration=0.939991501 podStartE2EDuration="3.91431861s" totalImagesPullingTime="2.974327109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:02:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:02:26.91360828 +0000 UTC m=+480.226481931" watchObservedRunningTime="2026-07-02 17:02:26.91431861 +0000 UTC m=+480.227192257" Jul 02 17:02:32.235080 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:32.235046 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9l27w"] Jul 02 17:02:32.237345 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:32.237328 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9l27w" Jul 02 17:02:32.239767 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:32.239750 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ln75z\"" Jul 02 17:02:32.245948 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:32.245923 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9l27w"] Jul 02 17:02:32.315309 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:32.315278 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8f75965f-2eb4-4e6d-a535-5ff873df05a9-bound-sa-token\") pod \"cert-manager-55b4d954d5-9l27w\" (UID: \"8f75965f-2eb4-4e6d-a535-5ff873df05a9\") " pod="cert-manager/cert-manager-55b4d954d5-9l27w" Jul 02 17:02:32.315450 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:32.315324 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2sjt\" (UniqueName: \"kubernetes.io/projected/8f75965f-2eb4-4e6d-a535-5ff873df05a9-kube-api-access-p2sjt\") pod \"cert-manager-55b4d954d5-9l27w\" (UID: \"8f75965f-2eb4-4e6d-a535-5ff873df05a9\") " pod="cert-manager/cert-manager-55b4d954d5-9l27w" Jul 02 17:02:32.416085 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:32.416051 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8f75965f-2eb4-4e6d-a535-5ff873df05a9-bound-sa-token\") pod \"cert-manager-55b4d954d5-9l27w\" (UID: \"8f75965f-2eb4-4e6d-a535-5ff873df05a9\") " pod="cert-manager/cert-manager-55b4d954d5-9l27w" Jul 02 17:02:32.416231 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:32.416122 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2sjt\" (UniqueName: \"kubernetes.io/projected/8f75965f-2eb4-4e6d-a535-5ff873df05a9-kube-api-access-p2sjt\") pod \"cert-manager-55b4d954d5-9l27w\" (UID: \"8f75965f-2eb4-4e6d-a535-5ff873df05a9\") " pod="cert-manager/cert-manager-55b4d954d5-9l27w" Jul 02 17:02:32.424260 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:32.424237 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8f75965f-2eb4-4e6d-a535-5ff873df05a9-bound-sa-token\") pod \"cert-manager-55b4d954d5-9l27w\" (UID: \"8f75965f-2eb4-4e6d-a535-5ff873df05a9\") " pod="cert-manager/cert-manager-55b4d954d5-9l27w" Jul 02 17:02:32.424382 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:32.424362 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2sjt\" (UniqueName: \"kubernetes.io/projected/8f75965f-2eb4-4e6d-a535-5ff873df05a9-kube-api-access-p2sjt\") pod \"cert-manager-55b4d954d5-9l27w\" (UID: \"8f75965f-2eb4-4e6d-a535-5ff873df05a9\") " pod="cert-manager/cert-manager-55b4d954d5-9l27w" Jul 02 17:02:32.546734 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:32.546690 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9l27w" Jul 02 17:02:32.676768 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:32.676739 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9l27w"] Jul 02 17:02:32.679784 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:02:32.679759 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f75965f_2eb4_4e6d_a535_5ff873df05a9.slice/crio-fbef363d36545f03ccf80a3425d1d9f52314e9e259608b3533e7c330e628e73c WatchSource:0}: Error finding container fbef363d36545f03ccf80a3425d1d9f52314e9e259608b3533e7c330e628e73c: Status 404 returned error can't find the container with id fbef363d36545f03ccf80a3425d1d9f52314e9e259608b3533e7c330e628e73c Jul 02 17:02:32.924483 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:32.924397 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9l27w" event={"ID":"8f75965f-2eb4-4e6d-a535-5ff873df05a9","Type":"ContainerStarted","Data":"cc0685d317773e051e79ca396abbb294db381681eb608a273882889a0aefbe52"} Jul 02 17:02:32.924483 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:32.924439 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9l27w" event={"ID":"8f75965f-2eb4-4e6d-a535-5ff873df05a9","Type":"ContainerStarted","Data":"fbef363d36545f03ccf80a3425d1d9f52314e9e259608b3533e7c330e628e73c"} Jul 02 17:02:32.939780 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:32.939709 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-9l27w" podStartSLOduration=0.939695759 podStartE2EDuration="939.695759ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:02:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:02:32.938418065 +0000 UTC m=+486.251291712" watchObservedRunningTime="2026-07-02 17:02:32.939695759 +0000 UTC m=+486.252569405" Jul 02 17:02:59.138118 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.138080 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck"] Jul 02 17:02:59.144717 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.144693 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.147355 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.147332 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 17:02:59.147489 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.147376 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 17:02:59.147604 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.147589 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k2nc7\"" Jul 02 17:02:59.147654 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.147609 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 17:02:59.147717 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.147701 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 17:02:59.154038 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.154012 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck"] Jul 02 17:02:59.231819 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.231794 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6e3d5582-387e-4bbd-a694-a63b3b65bf2e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-845f459c56-6skck\" (UID: \"6e3d5582-387e-4bbd-a694-a63b3b65bf2e\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.231819 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.231823 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2vbr\" (UniqueName: \"kubernetes.io/projected/6e3d5582-387e-4bbd-a694-a63b3b65bf2e-kube-api-access-p2vbr\") pod \"opendatahub-operator-controller-manager-845f459c56-6skck\" (UID: \"6e3d5582-387e-4bbd-a694-a63b3b65bf2e\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.231991 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.231846 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e3d5582-387e-4bbd-a694-a63b3b65bf2e-tmp\") pod \"opendatahub-operator-controller-manager-845f459c56-6skck\" (UID: \"6e3d5582-387e-4bbd-a694-a63b3b65bf2e\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.231991 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.231875 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e3d5582-387e-4bbd-a694-a63b3b65bf2e-webhook-cert\") pod \"opendatahub-operator-controller-manager-845f459c56-6skck\" (UID: \"6e3d5582-387e-4bbd-a694-a63b3b65bf2e\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.231991 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.231938 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6e3d5582-387e-4bbd-a694-a63b3b65bf2e-manifests\") pod \"opendatahub-operator-controller-manager-845f459c56-6skck\" (UID: \"6e3d5582-387e-4bbd-a694-a63b3b65bf2e\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.332822 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.332797 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2vbr\" (UniqueName: \"kubernetes.io/projected/6e3d5582-387e-4bbd-a694-a63b3b65bf2e-kube-api-access-p2vbr\") pod \"opendatahub-operator-controller-manager-845f459c56-6skck\" (UID: \"6e3d5582-387e-4bbd-a694-a63b3b65bf2e\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.332930 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.332826 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e3d5582-387e-4bbd-a694-a63b3b65bf2e-tmp\") pod \"opendatahub-operator-controller-manager-845f459c56-6skck\" (UID: \"6e3d5582-387e-4bbd-a694-a63b3b65bf2e\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.332930 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.332847 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e3d5582-387e-4bbd-a694-a63b3b65bf2e-webhook-cert\") pod \"opendatahub-operator-controller-manager-845f459c56-6skck\" (UID: \"6e3d5582-387e-4bbd-a694-a63b3b65bf2e\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.332930 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.332881 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6e3d5582-387e-4bbd-a694-a63b3b65bf2e-manifests\") pod \"opendatahub-operator-controller-manager-845f459c56-6skck\" (UID: \"6e3d5582-387e-4bbd-a694-a63b3b65bf2e\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.333046 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.332950 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6e3d5582-387e-4bbd-a694-a63b3b65bf2e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-845f459c56-6skck\" (UID: \"6e3d5582-387e-4bbd-a694-a63b3b65bf2e\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.333307 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.333282 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6e3d5582-387e-4bbd-a694-a63b3b65bf2e-manifests\") pod \"opendatahub-operator-controller-manager-845f459c56-6skck\" (UID: \"6e3d5582-387e-4bbd-a694-a63b3b65bf2e\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.335178 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.335137 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e3d5582-387e-4bbd-a694-a63b3b65bf2e-tmp\") pod \"opendatahub-operator-controller-manager-845f459c56-6skck\" (UID: \"6e3d5582-387e-4bbd-a694-a63b3b65bf2e\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.335405 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.335388 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6e3d5582-387e-4bbd-a694-a63b3b65bf2e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-845f459c56-6skck\" (UID: \"6e3d5582-387e-4bbd-a694-a63b3b65bf2e\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.335472 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.335408 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e3d5582-387e-4bbd-a694-a63b3b65bf2e-webhook-cert\") pod \"opendatahub-operator-controller-manager-845f459c56-6skck\" (UID: \"6e3d5582-387e-4bbd-a694-a63b3b65bf2e\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.341534 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.341515 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2vbr\" (UniqueName: \"kubernetes.io/projected/6e3d5582-387e-4bbd-a694-a63b3b65bf2e-kube-api-access-p2vbr\") pod \"opendatahub-operator-controller-manager-845f459c56-6skck\" (UID: \"6e3d5582-387e-4bbd-a694-a63b3b65bf2e\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.456293 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.456219 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:02:59.592042 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:02:59.592018 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck"] Jul 02 17:02:59.594332 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:02:59.594305 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e3d5582_387e_4bbd_a694_a63b3b65bf2e.slice/crio-5179336ba78469b07d01bdf341c78d7c974c4ba41ebf377d59cef20120448f33 WatchSource:0}: Error finding container 5179336ba78469b07d01bdf341c78d7c974c4ba41ebf377d59cef20120448f33: Status 404 returned error can't find the container with id 5179336ba78469b07d01bdf341c78d7c974c4ba41ebf377d59cef20120448f33 Jul 02 17:03:00.034943 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:00.034908 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" event={"ID":"6e3d5582-387e-4bbd-a694-a63b3b65bf2e","Type":"ContainerStarted","Data":"5179336ba78469b07d01bdf341c78d7c974c4ba41ebf377d59cef20120448f33"} Jul 02 17:03:04.054736 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:04.054700 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" event={"ID":"6e3d5582-387e-4bbd-a694-a63b3b65bf2e","Type":"ContainerStarted","Data":"e4a2786505609a32d25f1e034c0471a3f14d1fc4bb67eccfd96b48c5d846259f"} Jul 02 17:03:05.060762 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:05.060727 2556 generic.go:356] "Generic (PLEG): container finished" podID="6e3d5582-387e-4bbd-a694-a63b3b65bf2e" containerID="e4a2786505609a32d25f1e034c0471a3f14d1fc4bb67eccfd96b48c5d846259f" exitCode=0 Jul 02 17:03:05.061238 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:05.060763 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" event={"ID":"6e3d5582-387e-4bbd-a694-a63b3b65bf2e","Type":"ContainerDied","Data":"e4a2786505609a32d25f1e034c0471a3f14d1fc4bb67eccfd96b48c5d846259f"} Jul 02 17:03:06.067787 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:06.067755 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" event={"ID":"6e3d5582-387e-4bbd-a694-a63b3b65bf2e","Type":"ContainerStarted","Data":"68c9252010a6d1e90647a06fd85fec45fa0e367a0578ff99363e5c01f0f17ee5"} Jul 02 17:03:06.068183 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:06.067860 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:03:06.090179 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:06.090124 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" podStartSLOduration=2.125276539 podStartE2EDuration="7.090111021s" totalImagesPullingTime="3.958467723s" totalInitContainerRuntime="1.006366759s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:02:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:03:06.088732887 +0000 UTC m=+519.401606533" watchObservedRunningTime="2026-07-02 17:03:06.090111021 +0000 UTC m=+519.402984666" Jul 02 17:03:17.074552 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:17.074510 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-6skck" Jul 02 17:03:19.733812 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.733738 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf"] Jul 02 17:03:19.741845 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.741826 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf" Jul 02 17:03:19.745898 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.745876 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 02 17:03:19.746022 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.745894 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 17:03:19.747412 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.747391 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 02 17:03:19.747563 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.747417 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gpqz8\"" Jul 02 17:03:19.747563 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.747486 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 17:03:19.760799 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.760774 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf"] Jul 02 17:03:19.787418 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.787387 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58k65\" (UniqueName: \"kubernetes.io/projected/60a61adf-bf87-41bb-b5e2-631658ab2267-kube-api-access-58k65\") pod \"kube-auth-proxy-7bc8ccd95b-tt5nf\" (UID: \"60a61adf-bf87-41bb-b5e2-631658ab2267\") " pod="openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf" Jul 02 17:03:19.787581 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.787452 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60a61adf-bf87-41bb-b5e2-631658ab2267-tls-certs\") pod \"kube-auth-proxy-7bc8ccd95b-tt5nf\" (UID: \"60a61adf-bf87-41bb-b5e2-631658ab2267\") " pod="openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf" Jul 02 17:03:19.787581 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.787484 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60a61adf-bf87-41bb-b5e2-631658ab2267-tmp\") pod \"kube-auth-proxy-7bc8ccd95b-tt5nf\" (UID: \"60a61adf-bf87-41bb-b5e2-631658ab2267\") " pod="openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf" Jul 02 17:03:19.855028 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.854993 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zwblr"] Jul 02 17:03:19.858270 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.858256 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-zwblr" Jul 02 17:03:19.861819 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.861789 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-mvknj\"" Jul 02 17:03:19.862022 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.862008 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 02 17:03:19.888328 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.888303 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60a61adf-bf87-41bb-b5e2-631658ab2267-tls-certs\") pod \"kube-auth-proxy-7bc8ccd95b-tt5nf\" (UID: \"60a61adf-bf87-41bb-b5e2-631658ab2267\") " pod="openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf" Jul 02 17:03:19.888487 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.888340 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60a61adf-bf87-41bb-b5e2-631658ab2267-tmp\") pod \"kube-auth-proxy-7bc8ccd95b-tt5nf\" (UID: \"60a61adf-bf87-41bb-b5e2-631658ab2267\") " pod="openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf" Jul 02 17:03:19.888487 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.888365 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b660efa9-505d-4c85-9dba-7f86e21ea744-cert\") pod \"odh-model-controller-7998c67dc4-zwblr\" (UID: \"b660efa9-505d-4c85-9dba-7f86e21ea744\") " pod="opendatahub/odh-model-controller-7998c67dc4-zwblr" Jul 02 17:03:19.888487 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.888424 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58k65\" (UniqueName: \"kubernetes.io/projected/60a61adf-bf87-41bb-b5e2-631658ab2267-kube-api-access-58k65\") pod \"kube-auth-proxy-7bc8ccd95b-tt5nf\" (UID: \"60a61adf-bf87-41bb-b5e2-631658ab2267\") " pod="openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf" Jul 02 17:03:19.888487 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.888459 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tk5v\" (UniqueName: \"kubernetes.io/projected/b660efa9-505d-4c85-9dba-7f86e21ea744-kube-api-access-8tk5v\") pod \"odh-model-controller-7998c67dc4-zwblr\" (UID: \"b660efa9-505d-4c85-9dba-7f86e21ea744\") " pod="opendatahub/odh-model-controller-7998c67dc4-zwblr" Jul 02 17:03:19.888487 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:03:19.888464 2556 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 02 17:03:19.888735 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:03:19.888536 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60a61adf-bf87-41bb-b5e2-631658ab2267-tls-certs podName:60a61adf-bf87-41bb-b5e2-631658ab2267 nodeName:}" failed. No retries permitted until 2026-07-02 17:03:20.388515926 +0000 UTC m=+533.701389557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/60a61adf-bf87-41bb-b5e2-631658ab2267-tls-certs") pod "kube-auth-proxy-7bc8ccd95b-tt5nf" (UID: "60a61adf-bf87-41bb-b5e2-631658ab2267") : secret "kube-auth-proxy-tls" not found Jul 02 17:03:19.888905 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.888886 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zwblr"] Jul 02 17:03:19.890667 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.890645 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60a61adf-bf87-41bb-b5e2-631658ab2267-tmp\") pod \"kube-auth-proxy-7bc8ccd95b-tt5nf\" (UID: \"60a61adf-bf87-41bb-b5e2-631658ab2267\") " pod="openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf" Jul 02 17:03:19.912599 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.912579 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58k65\" (UniqueName: \"kubernetes.io/projected/60a61adf-bf87-41bb-b5e2-631658ab2267-kube-api-access-58k65\") pod \"kube-auth-proxy-7bc8ccd95b-tt5nf\" (UID: \"60a61adf-bf87-41bb-b5e2-631658ab2267\") " pod="openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf" Jul 02 17:03:19.989856 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.989772 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tk5v\" (UniqueName: \"kubernetes.io/projected/b660efa9-505d-4c85-9dba-7f86e21ea744-kube-api-access-8tk5v\") pod \"odh-model-controller-7998c67dc4-zwblr\" (UID: \"b660efa9-505d-4c85-9dba-7f86e21ea744\") " pod="opendatahub/odh-model-controller-7998c67dc4-zwblr" Jul 02 17:03:19.989996 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:19.989853 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b660efa9-505d-4c85-9dba-7f86e21ea744-cert\") pod \"odh-model-controller-7998c67dc4-zwblr\" (UID: \"b660efa9-505d-4c85-9dba-7f86e21ea744\") " pod="opendatahub/odh-model-controller-7998c67dc4-zwblr" Jul 02 17:03:19.989996 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:03:19.989956 2556 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 02 17:03:19.990108 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:03:19.990011 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b660efa9-505d-4c85-9dba-7f86e21ea744-cert podName:b660efa9-505d-4c85-9dba-7f86e21ea744 nodeName:}" failed. No retries permitted until 2026-07-02 17:03:20.489994074 +0000 UTC m=+533.802867697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b660efa9-505d-4c85-9dba-7f86e21ea744-cert") pod "odh-model-controller-7998c67dc4-zwblr" (UID: "b660efa9-505d-4c85-9dba-7f86e21ea744") : secret "odh-model-controller-webhook-cert" not found Jul 02 17:03:20.001840 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.001807 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tk5v\" (UniqueName: \"kubernetes.io/projected/b660efa9-505d-4c85-9dba-7f86e21ea744-kube-api-access-8tk5v\") pod \"odh-model-controller-7998c67dc4-zwblr\" (UID: \"b660efa9-505d-4c85-9dba-7f86e21ea744\") " pod="opendatahub/odh-model-controller-7998c67dc4-zwblr" Jul 02 17:03:20.393526 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.393494 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60a61adf-bf87-41bb-b5e2-631658ab2267-tls-certs\") pod \"kube-auth-proxy-7bc8ccd95b-tt5nf\" (UID: \"60a61adf-bf87-41bb-b5e2-631658ab2267\") " pod="openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf" Jul 02 17:03:20.395798 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.395778 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60a61adf-bf87-41bb-b5e2-631658ab2267-tls-certs\") pod \"kube-auth-proxy-7bc8ccd95b-tt5nf\" (UID: \"60a61adf-bf87-41bb-b5e2-631658ab2267\") " pod="openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf" Jul 02 17:03:20.494003 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.493967 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b660efa9-505d-4c85-9dba-7f86e21ea744-cert\") pod \"odh-model-controller-7998c67dc4-zwblr\" (UID: \"b660efa9-505d-4c85-9dba-7f86e21ea744\") " pod="opendatahub/odh-model-controller-7998c67dc4-zwblr" Jul 02 17:03:20.496493 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.496462 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b660efa9-505d-4c85-9dba-7f86e21ea744-cert\") pod \"odh-model-controller-7998c67dc4-zwblr\" (UID: \"b660efa9-505d-4c85-9dba-7f86e21ea744\") " pod="opendatahub/odh-model-controller-7998c67dc4-zwblr" Jul 02 17:03:20.651659 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.651580 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf" Jul 02 17:03:20.723653 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.723619 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-m984c"] Jul 02 17:03:20.728169 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.728135 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-m984c" Jul 02 17:03:20.731107 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.731084 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 02 17:03:20.740867 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.740835 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-m984c"] Jul 02 17:03:20.768361 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.768336 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-zwblr" Jul 02 17:03:20.793518 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.793461 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf"] Jul 02 17:03:20.796049 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:03:20.796020 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60a61adf_bf87_41bb_b5e2_631658ab2267.slice/crio-00b694ae7c5b459d132a456816573eed3815a4adae0b715837623187a2a2ee78 WatchSource:0}: Error finding container 00b694ae7c5b459d132a456816573eed3815a4adae0b715837623187a2a2ee78: Status 404 returned error can't find the container with id 00b694ae7c5b459d132a456816573eed3815a4adae0b715837623187a2a2ee78 Jul 02 17:03:20.796318 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.796300 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4f673c6-32fb-4165-be0f-d398f6aa2964-tls-certs\") pod \"model-serving-api-54d89bd5fc-m984c\" (UID: \"b4f673c6-32fb-4165-be0f-d398f6aa2964\") " pod="opendatahub/model-serving-api-54d89bd5fc-m984c" Jul 02 17:03:20.796389 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.796333 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxb9b\" (UniqueName: \"kubernetes.io/projected/b4f673c6-32fb-4165-be0f-d398f6aa2964-kube-api-access-pxb9b\") pod \"model-serving-api-54d89bd5fc-m984c\" (UID: \"b4f673c6-32fb-4165-be0f-d398f6aa2964\") " pod="opendatahub/model-serving-api-54d89bd5fc-m984c" Jul 02 17:03:20.895393 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.895367 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-zwblr"] Jul 02 17:03:20.896726 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.896706 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4f673c6-32fb-4165-be0f-d398f6aa2964-tls-certs\") pod \"model-serving-api-54d89bd5fc-m984c\" (UID: \"b4f673c6-32fb-4165-be0f-d398f6aa2964\") " pod="opendatahub/model-serving-api-54d89bd5fc-m984c" Jul 02 17:03:20.896815 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.896751 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxb9b\" (UniqueName: \"kubernetes.io/projected/b4f673c6-32fb-4165-be0f-d398f6aa2964-kube-api-access-pxb9b\") pod \"model-serving-api-54d89bd5fc-m984c\" (UID: \"b4f673c6-32fb-4165-be0f-d398f6aa2964\") " pod="opendatahub/model-serving-api-54d89bd5fc-m984c" Jul 02 17:03:20.896895 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:03:20.896867 2556 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 02 17:03:20.896944 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:03:20.896937 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f673c6-32fb-4165-be0f-d398f6aa2964-tls-certs podName:b4f673c6-32fb-4165-be0f-d398f6aa2964 nodeName:}" failed. No retries permitted until 2026-07-02 17:03:21.396919643 +0000 UTC m=+534.709793289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b4f673c6-32fb-4165-be0f-d398f6aa2964-tls-certs") pod "model-serving-api-54d89bd5fc-m984c" (UID: "b4f673c6-32fb-4165-be0f-d398f6aa2964") : secret "model-serving-api-tls" not found Jul 02 17:03:20.897402 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:03:20.897373 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb660efa9_505d_4c85_9dba_7f86e21ea744.slice/crio-e22e235e7534d9182d923a1b6770fa54b4bc5c7c843ea81ee0610a0899c41332 WatchSource:0}: Error finding container e22e235e7534d9182d923a1b6770fa54b4bc5c7c843ea81ee0610a0899c41332: Status 404 returned error can't find the container with id e22e235e7534d9182d923a1b6770fa54b4bc5c7c843ea81ee0610a0899c41332 Jul 02 17:03:20.912998 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.912946 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxb9b\" (UniqueName: \"kubernetes.io/projected/b4f673c6-32fb-4165-be0f-d398f6aa2964-kube-api-access-pxb9b\") pod \"model-serving-api-54d89bd5fc-m984c\" (UID: \"b4f673c6-32fb-4165-be0f-d398f6aa2964\") " pod="opendatahub/model-serving-api-54d89bd5fc-m984c" Jul 02 17:03:20.981779 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.981746 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b7crs"] Jul 02 17:03:20.986363 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.986346 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:20.988770 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.988747 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 02 17:03:20.988892 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.988874 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 02 17:03:20.988969 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.988956 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 02 17:03:20.989284 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.989267 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fc9hw\"" Jul 02 17:03:20.989595 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.989579 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 02 17:03:20.999803 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:20.999785 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b7crs"] Jul 02 17:03:21.097924 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.097886 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqqs6\" (UniqueName: \"kubernetes.io/projected/44220478-2bf6-4c18-b0a5-eec2f9535fb5-kube-api-access-cqqs6\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.098089 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.097934 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.098089 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.098008 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.098089 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.098055 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.098089 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.098076 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-cacerts\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.098339 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.098135 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/44220478-2bf6-4c18-b0a5-eec2f9535fb5-local-certs\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.098339 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.098189 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-token\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.133636 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.133606 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf" event={"ID":"60a61adf-bf87-41bb-b5e2-631658ab2267","Type":"ContainerStarted","Data":"00b694ae7c5b459d132a456816573eed3815a4adae0b715837623187a2a2ee78"} Jul 02 17:03:21.134747 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.134727 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-zwblr" event={"ID":"b660efa9-505d-4c85-9dba-7f86e21ea744","Type":"ContainerStarted","Data":"e22e235e7534d9182d923a1b6770fa54b4bc5c7c843ea81ee0610a0899c41332"} Jul 02 17:03:21.199488 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.199424 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.199488 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.199469 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.199640 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.199495 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-cacerts\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.199640 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.199531 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/44220478-2bf6-4c18-b0a5-eec2f9535fb5-local-certs\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.199640 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.199548 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-token\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.199640 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.199599 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqqs6\" (UniqueName: \"kubernetes.io/projected/44220478-2bf6-4c18-b0a5-eec2f9535fb5-kube-api-access-cqqs6\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.199640 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.199636 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.200201 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.200155 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.202030 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.201998 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-cacerts\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.202030 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.202023 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.202274 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.202258 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/44220478-2bf6-4c18-b0a5-eec2f9535fb5-local-certs\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.202457 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.202438 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.207423 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.207404 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-token\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.207765 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.207745 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqqs6\" (UniqueName: \"kubernetes.io/projected/44220478-2bf6-4c18-b0a5-eec2f9535fb5-kube-api-access-cqqs6\") pod \"istiod-openshift-gateway-7574459656-b7crs\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.296067 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.296031 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:21.401778 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.401736 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4f673c6-32fb-4165-be0f-d398f6aa2964-tls-certs\") pod \"model-serving-api-54d89bd5fc-m984c\" (UID: \"b4f673c6-32fb-4165-be0f-d398f6aa2964\") " pod="opendatahub/model-serving-api-54d89bd5fc-m984c" Jul 02 17:03:21.404180 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.404131 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4f673c6-32fb-4165-be0f-d398f6aa2964-tls-certs\") pod \"model-serving-api-54d89bd5fc-m984c\" (UID: \"b4f673c6-32fb-4165-be0f-d398f6aa2964\") " pod="opendatahub/model-serving-api-54d89bd5fc-m984c" Jul 02 17:03:21.422960 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.422934 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b7crs"] Jul 02 17:03:21.426644 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:03:21.426624 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44220478_2bf6_4c18_b0a5_eec2f9535fb5.slice/crio-f281c960b9a71be7cb7c4fe2b0382c7860239ec24f227eb4011c3da8978b4d34 WatchSource:0}: Error finding container f281c960b9a71be7cb7c4fe2b0382c7860239ec24f227eb4011c3da8978b4d34: Status 404 returned error can't find the container with id f281c960b9a71be7cb7c4fe2b0382c7860239ec24f227eb4011c3da8978b4d34 Jul 02 17:03:21.647771 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.647741 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-m984c" Jul 02 17:03:21.799773 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:21.799687 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-m984c"] Jul 02 17:03:21.805099 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:03:21.805070 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4f673c6_32fb_4165_be0f_d398f6aa2964.slice/crio-d2765fda122d0945c42b18ec09c456cf274e914bc8df8eee1f1b5ca9c163363e WatchSource:0}: Error finding container d2765fda122d0945c42b18ec09c456cf274e914bc8df8eee1f1b5ca9c163363e: Status 404 returned error can't find the container with id d2765fda122d0945c42b18ec09c456cf274e914bc8df8eee1f1b5ca9c163363e Jul 02 17:03:22.141832 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:22.141795 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-m984c" event={"ID":"b4f673c6-32fb-4165-be0f-d398f6aa2964","Type":"ContainerStarted","Data":"d2765fda122d0945c42b18ec09c456cf274e914bc8df8eee1f1b5ca9c163363e"} Jul 02 17:03:22.143084 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:22.143053 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" event={"ID":"44220478-2bf6-4c18-b0a5-eec2f9535fb5","Type":"ContainerStarted","Data":"f281c960b9a71be7cb7c4fe2b0382c7860239ec24f227eb4011c3da8978b4d34"} Jul 02 17:03:26.514941 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:26.514904 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:03:26.515321 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:26.514983 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:03:27.169496 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.169460 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-m984c" event={"ID":"b4f673c6-32fb-4165-be0f-d398f6aa2964","Type":"ContainerStarted","Data":"2cd0ad1408396651150904f0ae055f61482f77128810d577e1367cbb630a6556"} Jul 02 17:03:27.169690 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.169568 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-m984c" Jul 02 17:03:27.171081 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.171057 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf" event={"ID":"60a61adf-bf87-41bb-b5e2-631658ab2267","Type":"ContainerStarted","Data":"c940622f3ba43fc84f31b0ec91a0a9d629e8455483f7056b6cce580d57db9471"} Jul 02 17:03:27.172544 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.172525 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-zwblr" event={"ID":"b660efa9-505d-4c85-9dba-7f86e21ea744","Type":"ContainerStarted","Data":"02e84a3e0a0cb42443f97fb7d6d2283a7a344a69153d4271d642aec9d6873b5e"} Jul 02 17:03:27.172667 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.172653 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-zwblr" Jul 02 17:03:27.174076 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.174054 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" event={"ID":"44220478-2bf6-4c18-b0a5-eec2f9535fb5","Type":"ContainerStarted","Data":"43e32fcec8000cc7a9a86ebd90c0bc9645b3d1a348aba06838f7acf9f519f105"} Jul 02 17:03:27.174319 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.174293 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:27.176104 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.176084 2556 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-b7crs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 17:03:27.176222 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.176153 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" podUID="44220478-2bf6-4c18-b0a5-eec2f9535fb5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 17:03:27.187481 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.187441 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-m984c" podStartSLOduration=2.543953275 podStartE2EDuration="7.187427305s" totalImagesPullingTime="4.64347403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:03:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:03:27.186410675 +0000 UTC m=+540.499284323" watchObservedRunningTime="2026-07-02 17:03:27.187427305 +0000 UTC m=+540.500301132" Jul 02 17:03:27.205910 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.205860 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-zwblr" podStartSLOduration=2.653530423 podStartE2EDuration="8.205840786s" totalImagesPullingTime="5.552310363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:03:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:03:27.204311753 +0000 UTC m=+540.517185394" watchObservedRunningTime="2026-07-02 17:03:27.205840786 +0000 UTC m=+540.518714433" Jul 02 17:03:27.233869 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.233809 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" podStartSLOduration=2.147821026 podStartE2EDuration="7.233790704s" totalImagesPullingTime="5.085969678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:03:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:03:27.232912654 +0000 UTC m=+540.545786323" watchObservedRunningTime="2026-07-02 17:03:27.233790704 +0000 UTC m=+540.546664349" Jul 02 17:03:27.260409 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.260332 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7bc8ccd95b-tt5nf" podStartSLOduration=2.606886407 podStartE2EDuration="8.260312012s" totalImagesPullingTime="5.653425605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:03:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:03:27.257565185 +0000 UTC m=+540.570438833" watchObservedRunningTime="2026-07-02 17:03:27.260312012 +0000 UTC m=+540.573185660" Jul 02 17:03:27.840253 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.839986 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22"] Jul 02 17:03:27.846056 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.846022 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:27.848950 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.848927 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-dp92b\"" Jul 02 17:03:27.859056 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.859027 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22"] Jul 02 17:03:27.962736 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.962700 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bad9014-0315-437b-a0d6-7504cd44aca8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:27.962900 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.962746 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bad9014-0315-437b-a0d6-7504cd44aca8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:27.962900 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.962811 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bad9014-0315-437b-a0d6-7504cd44aca8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:27.962900 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.962843 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bad9014-0315-437b-a0d6-7504cd44aca8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:27.962900 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.962864 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bad9014-0315-437b-a0d6-7504cd44aca8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:27.962900 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.962883 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sljgp\" (UniqueName: \"kubernetes.io/projected/7bad9014-0315-437b-a0d6-7504cd44aca8-kube-api-access-sljgp\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:27.963057 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.962983 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bad9014-0315-437b-a0d6-7504cd44aca8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:27.963057 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.963018 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bad9014-0315-437b-a0d6-7504cd44aca8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:27.963057 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:27.963042 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bad9014-0315-437b-a0d6-7504cd44aca8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.064131 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.064083 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bad9014-0315-437b-a0d6-7504cd44aca8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.064343 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.064145 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bad9014-0315-437b-a0d6-7504cd44aca8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.064343 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.064195 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sljgp\" (UniqueName: \"kubernetes.io/projected/7bad9014-0315-437b-a0d6-7504cd44aca8-kube-api-access-sljgp\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.064343 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.064243 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bad9014-0315-437b-a0d6-7504cd44aca8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.064343 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.064267 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bad9014-0315-437b-a0d6-7504cd44aca8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.064343 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.064300 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bad9014-0315-437b-a0d6-7504cd44aca8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.064343 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.064336 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bad9014-0315-437b-a0d6-7504cd44aca8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.064643 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.064373 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bad9014-0315-437b-a0d6-7504cd44aca8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.064643 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.064443 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bad9014-0315-437b-a0d6-7504cd44aca8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.064792 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.064744 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bad9014-0315-437b-a0d6-7504cd44aca8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.064844 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.064793 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bad9014-0315-437b-a0d6-7504cd44aca8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.065190 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.065149 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bad9014-0315-437b-a0d6-7504cd44aca8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.065475 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.065449 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bad9014-0315-437b-a0d6-7504cd44aca8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.065633 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.065612 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bad9014-0315-437b-a0d6-7504cd44aca8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.068559 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.067971 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bad9014-0315-437b-a0d6-7504cd44aca8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.068559 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.068509 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bad9014-0315-437b-a0d6-7504cd44aca8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.074681 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.074656 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bad9014-0315-437b-a0d6-7504cd44aca8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.075114 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.075091 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sljgp\" (UniqueName: \"kubernetes.io/projected/7bad9014-0315-437b-a0d6-7504cd44aca8-kube-api-access-sljgp\") pod \"data-science-gateway-data-science-gateway-class-656c84dccdqgg22\" (UID: \"7bad9014-0315-437b-a0d6-7504cd44aca8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.156862 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.156755 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:28.180270 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.180241 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:28.328136 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:28.328094 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22"] Jul 02 17:03:28.331616 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:03:28.331060 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bad9014_0315_437b_a0d6_7504cd44aca8.slice/crio-d7cc1072acf59ce576b49f54b87db886b386a6a5e869f3ab5bffd037ce4b7c37 WatchSource:0}: Error finding container d7cc1072acf59ce576b49f54b87db886b386a6a5e869f3ab5bffd037ce4b7c37: Status 404 returned error can't find the container with id d7cc1072acf59ce576b49f54b87db886b386a6a5e869f3ab5bffd037ce4b7c37 Jul 02 17:03:29.188853 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:29.188817 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" event={"ID":"7bad9014-0315-437b-a0d6-7504cd44aca8","Type":"ContainerStarted","Data":"d7cc1072acf59ce576b49f54b87db886b386a6a5e869f3ab5bffd037ce4b7c37"} Jul 02 17:03:30.597033 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.596997 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-d6gv7"] Jul 02 17:03:30.600931 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.600905 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6gv7" Jul 02 17:03:30.603519 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.603445 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 02 17:03:30.603519 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.603494 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-44l92\"" Jul 02 17:03:30.607154 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.607132 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kfbp2"] Jul 02 17:03:30.611009 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.610985 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kfbp2" Jul 02 17:03:30.613801 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.613775 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4nm7m\"" Jul 02 17:03:30.613913 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.613840 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 02 17:03:30.617452 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.617432 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-d6gv7"] Jul 02 17:03:30.622957 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.622934 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kfbp2"] Jul 02 17:03:30.699698 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.699667 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwv28\" (UniqueName: \"kubernetes.io/projected/2f06eafc-832e-40c1-9117-de241dbf4209-kube-api-access-xwv28\") pod \"kserve-controller-manager-d6f4b65b8-kfbp2\" (UID: \"2f06eafc-832e-40c1-9117-de241dbf4209\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kfbp2" Jul 02 17:03:30.699852 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.699793 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36675e91-c4ed-46d0-ab14-43cf33c2216b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d6gv7\" (UID: \"36675e91-c4ed-46d0-ab14-43cf33c2216b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6gv7" Jul 02 17:03:30.699915 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.699860 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f06eafc-832e-40c1-9117-de241dbf4209-cert\") pod \"kserve-controller-manager-d6f4b65b8-kfbp2\" (UID: \"2f06eafc-832e-40c1-9117-de241dbf4209\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kfbp2" Jul 02 17:03:30.699915 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.699894 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twhsw\" (UniqueName: \"kubernetes.io/projected/36675e91-c4ed-46d0-ab14-43cf33c2216b-kube-api-access-twhsw\") pod \"llmisvc-controller-manager-777c6bd7d5-d6gv7\" (UID: \"36675e91-c4ed-46d0-ab14-43cf33c2216b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6gv7" Jul 02 17:03:30.800599 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.800563 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f06eafc-832e-40c1-9117-de241dbf4209-cert\") pod \"kserve-controller-manager-d6f4b65b8-kfbp2\" (UID: \"2f06eafc-832e-40c1-9117-de241dbf4209\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kfbp2" Jul 02 17:03:30.800813 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.800620 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twhsw\" (UniqueName: \"kubernetes.io/projected/36675e91-c4ed-46d0-ab14-43cf33c2216b-kube-api-access-twhsw\") pod \"llmisvc-controller-manager-777c6bd7d5-d6gv7\" (UID: \"36675e91-c4ed-46d0-ab14-43cf33c2216b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6gv7" Jul 02 17:03:30.800813 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.800697 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwv28\" (UniqueName: \"kubernetes.io/projected/2f06eafc-832e-40c1-9117-de241dbf4209-kube-api-access-xwv28\") pod \"kserve-controller-manager-d6f4b65b8-kfbp2\" (UID: \"2f06eafc-832e-40c1-9117-de241dbf4209\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kfbp2" Jul 02 17:03:30.800813 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.800776 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36675e91-c4ed-46d0-ab14-43cf33c2216b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d6gv7\" (UID: \"36675e91-c4ed-46d0-ab14-43cf33c2216b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6gv7" Jul 02 17:03:30.803885 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.803837 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36675e91-c4ed-46d0-ab14-43cf33c2216b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d6gv7\" (UID: \"36675e91-c4ed-46d0-ab14-43cf33c2216b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6gv7" Jul 02 17:03:30.804569 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.804545 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f06eafc-832e-40c1-9117-de241dbf4209-cert\") pod \"kserve-controller-manager-d6f4b65b8-kfbp2\" (UID: \"2f06eafc-832e-40c1-9117-de241dbf4209\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kfbp2" Jul 02 17:03:30.811448 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.811424 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwv28\" (UniqueName: \"kubernetes.io/projected/2f06eafc-832e-40c1-9117-de241dbf4209-kube-api-access-xwv28\") pod \"kserve-controller-manager-d6f4b65b8-kfbp2\" (UID: \"2f06eafc-832e-40c1-9117-de241dbf4209\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kfbp2" Jul 02 17:03:30.812248 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.812224 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twhsw\" (UniqueName: \"kubernetes.io/projected/36675e91-c4ed-46d0-ab14-43cf33c2216b-kube-api-access-twhsw\") pod \"llmisvc-controller-manager-777c6bd7d5-d6gv7\" (UID: \"36675e91-c4ed-46d0-ab14-43cf33c2216b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6gv7" Jul 02 17:03:30.914518 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.914439 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6gv7" Jul 02 17:03:30.926143 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:30.926120 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kfbp2" Jul 02 17:03:31.761187 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:31.759586 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:03:31.761187 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:31.759671 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:03:31.761187 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:31.759709 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:03:31.835517 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:31.835485 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-d6gv7"] Jul 02 17:03:31.847315 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:03:31.847293 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod36675e91_c4ed_46d0_ab14_43cf33c2216b.slice/crio-f14d42235db8389b99ebd0f0a6478a5983687dcfbe486cfbd23335ea0d762edc WatchSource:0}: Error finding container f14d42235db8389b99ebd0f0a6478a5983687dcfbe486cfbd23335ea0d762edc: Status 404 returned error can't find the container with id f14d42235db8389b99ebd0f0a6478a5983687dcfbe486cfbd23335ea0d762edc Jul 02 17:03:31.855314 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:31.855250 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kfbp2"] Jul 02 17:03:31.858244 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:03:31.858220 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f06eafc_832e_40c1_9117_de241dbf4209.slice/crio-0aaf782b34b5f67f107168422cb05a777e7b6e9bf78ea2aebf14ce3a857bf3a4 WatchSource:0}: Error finding container 0aaf782b34b5f67f107168422cb05a777e7b6e9bf78ea2aebf14ce3a857bf3a4: Status 404 returned error can't find the container with id 0aaf782b34b5f67f107168422cb05a777e7b6e9bf78ea2aebf14ce3a857bf3a4 Jul 02 17:03:32.206629 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:32.206584 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" event={"ID":"7bad9014-0315-437b-a0d6-7504cd44aca8","Type":"ContainerStarted","Data":"8a93f431b0576c1d1391ae12b3ddaf4baefaf5326d254b8f0379cd6d6aed603b"} Jul 02 17:03:32.207864 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:32.207836 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kfbp2" event={"ID":"2f06eafc-832e-40c1-9117-de241dbf4209","Type":"ContainerStarted","Data":"0aaf782b34b5f67f107168422cb05a777e7b6e9bf78ea2aebf14ce3a857bf3a4"} Jul 02 17:03:32.208980 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:32.208954 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6gv7" event={"ID":"36675e91-c4ed-46d0-ab14-43cf33c2216b","Type":"ContainerStarted","Data":"f14d42235db8389b99ebd0f0a6478a5983687dcfbe486cfbd23335ea0d762edc"} Jul 02 17:03:32.247440 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:32.247395 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" podStartSLOduration=1.8220323710000002 podStartE2EDuration="5.24738129s" totalImagesPullingTime="3.425348919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:03:32.244653739 +0000 UTC m=+545.557527385" watchObservedRunningTime="2026-07-02 17:03:32.24738129 +0000 UTC m=+545.560254935" Jul 02 17:03:32.580182 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:32.580120 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b7crs"] Jul 02 17:03:32.580421 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:32.580391 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" podUID="44220478-2bf6-4c18-b0a5-eec2f9535fb5" containerName="discovery" containerID="cri-o://43e32fcec8000cc7a9a86ebd90c0bc9645b3d1a348aba06838f7acf9f519f105" gracePeriod=30 Jul 02 17:03:32.997362 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:32.997334 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:33.122069 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.122036 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/44220478-2bf6-4c18-b0a5-eec2f9535fb5-local-certs\") pod \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " Jul 02 17:03:33.122270 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.122114 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-cacerts\") pod \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " Jul 02 17:03:33.122270 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.122180 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-csr-ca-configmap\") pod \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " Jul 02 17:03:33.122270 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.122252 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-csr-dns-cert\") pod \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " Jul 02 17:03:33.122499 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.122288 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-kubeconfig\") pod \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " Jul 02 17:03:33.122499 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.122323 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-token\") pod \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " Jul 02 17:03:33.122499 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.122353 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqqs6\" (UniqueName: \"kubernetes.io/projected/44220478-2bf6-4c18-b0a5-eec2f9535fb5-kube-api-access-cqqs6\") pod \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\" (UID: \"44220478-2bf6-4c18-b0a5-eec2f9535fb5\") " Jul 02 17:03:33.123294 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.123255 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "44220478-2bf6-4c18-b0a5-eec2f9535fb5" (UID: "44220478-2bf6-4c18-b0a5-eec2f9535fb5"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:03:33.124921 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.124862 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "44220478-2bf6-4c18-b0a5-eec2f9535fb5" (UID: "44220478-2bf6-4c18-b0a5-eec2f9535fb5"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:03:33.125546 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.125518 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44220478-2bf6-4c18-b0a5-eec2f9535fb5-kube-api-access-cqqs6" (OuterVolumeSpecName: "kube-api-access-cqqs6") pod "44220478-2bf6-4c18-b0a5-eec2f9535fb5" (UID: "44220478-2bf6-4c18-b0a5-eec2f9535fb5"). InnerVolumeSpecName "kube-api-access-cqqs6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:03:33.125803 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.125777 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-token" (OuterVolumeSpecName: "istio-token") pod "44220478-2bf6-4c18-b0a5-eec2f9535fb5" (UID: "44220478-2bf6-4c18-b0a5-eec2f9535fb5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:03:33.126097 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.126064 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-cacerts" (OuterVolumeSpecName: "cacerts") pod "44220478-2bf6-4c18-b0a5-eec2f9535fb5" (UID: "44220478-2bf6-4c18-b0a5-eec2f9535fb5"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:03:33.126250 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.126221 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44220478-2bf6-4c18-b0a5-eec2f9535fb5-local-certs" (OuterVolumeSpecName: "local-certs") pod "44220478-2bf6-4c18-b0a5-eec2f9535fb5" (UID: "44220478-2bf6-4c18-b0a5-eec2f9535fb5"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:03:33.126878 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.126852 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "44220478-2bf6-4c18-b0a5-eec2f9535fb5" (UID: "44220478-2bf6-4c18-b0a5-eec2f9535fb5"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:03:33.157235 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.157206 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:33.162664 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.162634 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:33.216178 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.216124 2556 generic.go:356] "Generic (PLEG): container finished" podID="44220478-2bf6-4c18-b0a5-eec2f9535fb5" containerID="43e32fcec8000cc7a9a86ebd90c0bc9645b3d1a348aba06838f7acf9f519f105" exitCode=0 Jul 02 17:03:33.216311 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.216193 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" event={"ID":"44220478-2bf6-4c18-b0a5-eec2f9535fb5","Type":"ContainerDied","Data":"43e32fcec8000cc7a9a86ebd90c0bc9645b3d1a348aba06838f7acf9f519f105"} Jul 02 17:03:33.216311 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.216257 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" event={"ID":"44220478-2bf6-4c18-b0a5-eec2f9535fb5","Type":"ContainerDied","Data":"f281c960b9a71be7cb7c4fe2b0382c7860239ec24f227eb4011c3da8978b4d34"} Jul 02 17:03:33.216311 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.216286 2556 scope.go:117] "RemoveContainer" containerID="43e32fcec8000cc7a9a86ebd90c0bc9645b3d1a348aba06838f7acf9f519f105" Jul 02 17:03:33.217078 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.216535 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-b7crs" Jul 02 17:03:33.217078 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.216785 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:33.217983 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.217963 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccdqgg22" Jul 02 17:03:33.223946 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.223924 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-token\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:03:33.224058 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.223950 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqqs6\" (UniqueName: \"kubernetes.io/projected/44220478-2bf6-4c18-b0a5-eec2f9535fb5-kube-api-access-cqqs6\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:03:33.224058 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.223965 2556 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/44220478-2bf6-4c18-b0a5-eec2f9535fb5-local-certs\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:03:33.224058 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.223980 2556 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-cacerts\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:03:33.224058 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.223993 2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-csr-ca-configmap\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:03:33.224058 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.224006 2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-csr-dns-cert\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:03:33.224058 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.224020 2556 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44220478-2bf6-4c18-b0a5-eec2f9535fb5-istio-kubeconfig\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:03:33.259309 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.259271 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b7crs"] Jul 02 17:03:33.264610 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.264553 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-b7crs"] Jul 02 17:03:33.308974 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.308953 2556 scope.go:117] "RemoveContainer" containerID="43e32fcec8000cc7a9a86ebd90c0bc9645b3d1a348aba06838f7acf9f519f105" Jul 02 17:03:33.309430 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:03:33.309398 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43e32fcec8000cc7a9a86ebd90c0bc9645b3d1a348aba06838f7acf9f519f105\": container with ID starting with 43e32fcec8000cc7a9a86ebd90c0bc9645b3d1a348aba06838f7acf9f519f105 not found: ID does not exist" containerID="43e32fcec8000cc7a9a86ebd90c0bc9645b3d1a348aba06838f7acf9f519f105" Jul 02 17:03:33.309590 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:33.309442 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43e32fcec8000cc7a9a86ebd90c0bc9645b3d1a348aba06838f7acf9f519f105"} err="failed to get container status \"43e32fcec8000cc7a9a86ebd90c0bc9645b3d1a348aba06838f7acf9f519f105\": rpc error: code = NotFound desc = could not find container \"43e32fcec8000cc7a9a86ebd90c0bc9645b3d1a348aba06838f7acf9f519f105\": container with ID starting with 43e32fcec8000cc7a9a86ebd90c0bc9645b3d1a348aba06838f7acf9f519f105 not found: ID does not exist" Jul 02 17:03:35.190728 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:35.190698 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44220478-2bf6-4c18-b0a5-eec2f9535fb5" path="/var/lib/kubelet/pods/44220478-2bf6-4c18-b0a5-eec2f9535fb5/volumes" Jul 02 17:03:36.235333 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:36.235294 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6gv7" event={"ID":"36675e91-c4ed-46d0-ab14-43cf33c2216b","Type":"ContainerStarted","Data":"2c0043ca0713b3800ea1693e2cbec72564a85a132403ef7c0eed654fd9a02083"} Jul 02 17:03:36.235794 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:36.235365 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6gv7" Jul 02 17:03:36.236696 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:36.236670 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kfbp2" event={"ID":"2f06eafc-832e-40c1-9117-de241dbf4209","Type":"ContainerStarted","Data":"9a2c6f9c692eefbfa57a19664aa53fb695e91535e533fcf9cd3cb75cc33e32d4"} Jul 02 17:03:36.236823 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:36.236801 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kfbp2" Jul 02 17:03:36.263966 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:36.263923 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6gv7" podStartSLOduration=2.774270233 podStartE2EDuration="6.263908788s" totalImagesPullingTime="3.489638555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:03:36.262684228 +0000 UTC m=+549.575557875" watchObservedRunningTime="2026-07-02 17:03:36.263908788 +0000 UTC m=+549.576782434" Jul 02 17:03:36.294950 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:36.294909 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kfbp2" podStartSLOduration=2.8201305 podStartE2EDuration="6.294895672s" totalImagesPullingTime="3.474765172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:03:36.293769587 +0000 UTC m=+549.606643234" watchObservedRunningTime="2026-07-02 17:03:36.294895672 +0000 UTC m=+549.607769317" Jul 02 17:03:38.181693 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:38.181658 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-zwblr" Jul 02 17:03:38.183695 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:03:38.183675 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-m984c" Jul 02 17:04:07.244573 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:07.244540 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d6gv7" Jul 02 17:04:07.247789 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:07.247768 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kfbp2" Jul 02 17:04:20.852624 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:20.852588 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8mzs6"] Jul 02 17:04:20.853213 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:20.853100 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44220478-2bf6-4c18-b0a5-eec2f9535fb5" containerName="discovery" Jul 02 17:04:20.853213 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:20.853128 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="44220478-2bf6-4c18-b0a5-eec2f9535fb5" containerName="discovery" Jul 02 17:04:20.853340 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:20.853299 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="44220478-2bf6-4c18-b0a5-eec2f9535fb5" containerName="discovery" Jul 02 17:04:20.855536 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:20.855518 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8mzs6" Jul 02 17:04:20.858508 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:20.858340 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-lrzdd\"" Jul 02 17:04:20.858508 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:20.858358 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 17:04:20.858729 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:20.858704 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8mzs6"] Jul 02 17:04:20.859398 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:20.859375 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 17:04:20.920861 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:20.920832 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbtdp\" (UniqueName: \"kubernetes.io/projected/10027c6d-967c-4474-b089-a778d52c9ec3-kube-api-access-wbtdp\") pod \"kuadrant-operator-catalog-8mzs6\" (UID: \"10027c6d-967c-4474-b089-a778d52c9ec3\") " pod="kuadrant-system/kuadrant-operator-catalog-8mzs6" Jul 02 17:04:21.022351 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:21.022316 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbtdp\" (UniqueName: \"kubernetes.io/projected/10027c6d-967c-4474-b089-a778d52c9ec3-kube-api-access-wbtdp\") pod \"kuadrant-operator-catalog-8mzs6\" (UID: \"10027c6d-967c-4474-b089-a778d52c9ec3\") " pod="kuadrant-system/kuadrant-operator-catalog-8mzs6" Jul 02 17:04:21.030211 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:21.030182 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbtdp\" (UniqueName: \"kubernetes.io/projected/10027c6d-967c-4474-b089-a778d52c9ec3-kube-api-access-wbtdp\") pod \"kuadrant-operator-catalog-8mzs6\" (UID: \"10027c6d-967c-4474-b089-a778d52c9ec3\") " pod="kuadrant-system/kuadrant-operator-catalog-8mzs6" Jul 02 17:04:21.166434 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:21.166341 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8mzs6" Jul 02 17:04:21.211310 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:21.211279 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8mzs6"] Jul 02 17:04:21.300217 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:21.300196 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8mzs6"] Jul 02 17:04:21.302689 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:04:21.302660 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10027c6d_967c_4474_b089_a778d52c9ec3.slice/crio-ceb0092b6cec6942cbd983f15a26873ea65712c7a4a2885282904c0732693105 WatchSource:0}: Error finding container ceb0092b6cec6942cbd983f15a26873ea65712c7a4a2885282904c0732693105: Status 404 returned error can't find the container with id ceb0092b6cec6942cbd983f15a26873ea65712c7a4a2885282904c0732693105 Jul 02 17:04:21.423463 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:21.423380 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r68bn"] Jul 02 17:04:21.426416 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:21.426398 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r68bn" Jul 02 17:04:21.433960 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:21.433929 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r68bn"] Jul 02 17:04:21.443346 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:21.443312 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8mzs6" event={"ID":"10027c6d-967c-4474-b089-a778d52c9ec3","Type":"ContainerStarted","Data":"ceb0092b6cec6942cbd983f15a26873ea65712c7a4a2885282904c0732693105"} Jul 02 17:04:21.527982 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:21.527946 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2blcx\" (UniqueName: \"kubernetes.io/projected/53532975-54fb-4f3e-bd1a-c6fff83395e3-kube-api-access-2blcx\") pod \"kuadrant-operator-catalog-r68bn\" (UID: \"53532975-54fb-4f3e-bd1a-c6fff83395e3\") " pod="kuadrant-system/kuadrant-operator-catalog-r68bn" Jul 02 17:04:21.628930 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:21.628877 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2blcx\" (UniqueName: \"kubernetes.io/projected/53532975-54fb-4f3e-bd1a-c6fff83395e3-kube-api-access-2blcx\") pod \"kuadrant-operator-catalog-r68bn\" (UID: \"53532975-54fb-4f3e-bd1a-c6fff83395e3\") " pod="kuadrant-system/kuadrant-operator-catalog-r68bn" Jul 02 17:04:21.637285 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:21.637260 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2blcx\" (UniqueName: \"kubernetes.io/projected/53532975-54fb-4f3e-bd1a-c6fff83395e3-kube-api-access-2blcx\") pod \"kuadrant-operator-catalog-r68bn\" (UID: \"53532975-54fb-4f3e-bd1a-c6fff83395e3\") " pod="kuadrant-system/kuadrant-operator-catalog-r68bn" Jul 02 17:04:21.736898 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:21.736822 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r68bn" Jul 02 17:04:21.901211 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:21.901176 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r68bn"] Jul 02 17:04:21.972633 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:04:21.972594 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53532975_54fb_4f3e_bd1a_c6fff83395e3.slice/crio-d9b7b2169b9143a11fcfefc05f03633a9ed51e5bf0a11e9352086a51dd591217 WatchSource:0}: Error finding container d9b7b2169b9143a11fcfefc05f03633a9ed51e5bf0a11e9352086a51dd591217: Status 404 returned error can't find the container with id d9b7b2169b9143a11fcfefc05f03633a9ed51e5bf0a11e9352086a51dd591217 Jul 02 17:04:22.452347 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:22.452296 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r68bn" event={"ID":"53532975-54fb-4f3e-bd1a-c6fff83395e3","Type":"ContainerStarted","Data":"d9b7b2169b9143a11fcfefc05f03633a9ed51e5bf0a11e9352086a51dd591217"} Jul 02 17:04:24.463093 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:24.463050 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8mzs6" event={"ID":"10027c6d-967c-4474-b089-a778d52c9ec3","Type":"ContainerStarted","Data":"9a9606700e7c80864db5453111ce57d22233f237d9c4b0b4b03012c8b25a6096"} Jul 02 17:04:24.463537 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:24.463173 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-8mzs6" podUID="10027c6d-967c-4474-b089-a778d52c9ec3" containerName="registry-server" containerID="cri-o://9a9606700e7c80864db5453111ce57d22233f237d9c4b0b4b03012c8b25a6096" gracePeriod=2 Jul 02 17:04:24.464647 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:24.464613 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r68bn" event={"ID":"53532975-54fb-4f3e-bd1a-c6fff83395e3","Type":"ContainerStarted","Data":"142ea020af9f061ee4500cb95fc053bf6e46b843966e06287bcd04070adfeb61"} Jul 02 17:04:24.480092 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:24.480048 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-8mzs6" podStartSLOduration=2.263545589 podStartE2EDuration="4.480032867s" totalImagesPullingTime="2.216487278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:04:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:04:24.479272437 +0000 UTC m=+597.792146083" watchObservedRunningTime="2026-07-02 17:04:24.480032867 +0000 UTC m=+597.792906514" Jul 02 17:04:24.496040 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:24.495993 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-r68bn" podStartSLOduration=1.94645829 podStartE2EDuration="3.495982101s" totalImagesPullingTime="1.549523811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:04:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:04:24.494414054 +0000 UTC m=+597.807287701" watchObservedRunningTime="2026-07-02 17:04:24.495982101 +0000 UTC m=+597.808855747" Jul 02 17:04:24.715974 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:24.715915 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8mzs6" Jul 02 17:04:24.858328 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:24.858290 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbtdp\" (UniqueName: \"kubernetes.io/projected/10027c6d-967c-4474-b089-a778d52c9ec3-kube-api-access-wbtdp\") pod \"10027c6d-967c-4474-b089-a778d52c9ec3\" (UID: \"10027c6d-967c-4474-b089-a778d52c9ec3\") " Jul 02 17:04:24.860448 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:24.860421 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10027c6d-967c-4474-b089-a778d52c9ec3-kube-api-access-wbtdp" (OuterVolumeSpecName: "kube-api-access-wbtdp") pod "10027c6d-967c-4474-b089-a778d52c9ec3" (UID: "10027c6d-967c-4474-b089-a778d52c9ec3"). InnerVolumeSpecName "kube-api-access-wbtdp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:04:24.959874 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:24.959840 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbtdp\" (UniqueName: \"kubernetes.io/projected/10027c6d-967c-4474-b089-a778d52c9ec3-kube-api-access-wbtdp\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:04:25.470137 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:25.470045 2556 generic.go:356] "Generic (PLEG): container finished" podID="10027c6d-967c-4474-b089-a778d52c9ec3" containerID="9a9606700e7c80864db5453111ce57d22233f237d9c4b0b4b03012c8b25a6096" exitCode=0 Jul 02 17:04:25.470137 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:25.470090 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8mzs6" Jul 02 17:04:25.470708 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:25.470135 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8mzs6" event={"ID":"10027c6d-967c-4474-b089-a778d52c9ec3","Type":"ContainerDied","Data":"9a9606700e7c80864db5453111ce57d22233f237d9c4b0b4b03012c8b25a6096"} Jul 02 17:04:25.470708 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:25.470211 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8mzs6" event={"ID":"10027c6d-967c-4474-b089-a778d52c9ec3","Type":"ContainerDied","Data":"ceb0092b6cec6942cbd983f15a26873ea65712c7a4a2885282904c0732693105"} Jul 02 17:04:25.470708 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:25.470236 2556 scope.go:117] "RemoveContainer" containerID="9a9606700e7c80864db5453111ce57d22233f237d9c4b0b4b03012c8b25a6096" Jul 02 17:04:25.479641 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:25.479624 2556 scope.go:117] "RemoveContainer" containerID="9a9606700e7c80864db5453111ce57d22233f237d9c4b0b4b03012c8b25a6096" Jul 02 17:04:25.479905 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:04:25.479889 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a9606700e7c80864db5453111ce57d22233f237d9c4b0b4b03012c8b25a6096\": container with ID starting with 9a9606700e7c80864db5453111ce57d22233f237d9c4b0b4b03012c8b25a6096 not found: ID does not exist" containerID="9a9606700e7c80864db5453111ce57d22233f237d9c4b0b4b03012c8b25a6096" Jul 02 17:04:25.479961 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:25.479912 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a9606700e7c80864db5453111ce57d22233f237d9c4b0b4b03012c8b25a6096"} err="failed to get container status \"9a9606700e7c80864db5453111ce57d22233f237d9c4b0b4b03012c8b25a6096\": rpc error: code = NotFound desc = could not find container \"9a9606700e7c80864db5453111ce57d22233f237d9c4b0b4b03012c8b25a6096\": container with ID starting with 9a9606700e7c80864db5453111ce57d22233f237d9c4b0b4b03012c8b25a6096 not found: ID does not exist" Jul 02 17:04:25.485445 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:25.485419 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8mzs6"] Jul 02 17:04:25.487504 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:25.487483 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8mzs6"] Jul 02 17:04:27.144471 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:27.144390 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 17:04:27.144910 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:27.144890 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 17:04:27.189417 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:27.189385 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10027c6d-967c-4474-b089-a778d52c9ec3" path="/var/lib/kubelet/pods/10027c6d-967c-4474-b089-a778d52c9ec3/volumes" Jul 02 17:04:31.737952 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:31.737902 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-r68bn" Jul 02 17:04:31.740420 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:31.737980 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-r68bn" Jul 02 17:04:31.761062 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:31.761028 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-r68bn" Jul 02 17:04:32.527108 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:32.527077 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-r68bn" Jul 02 17:04:52.390602 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:52.390510 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lqrmq"] Jul 02 17:04:52.391023 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:52.390859 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10027c6d-967c-4474-b089-a778d52c9ec3" containerName="registry-server" Jul 02 17:04:52.391023 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:52.390869 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="10027c6d-967c-4474-b089-a778d52c9ec3" containerName="registry-server" Jul 02 17:04:52.391023 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:52.390944 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="10027c6d-967c-4474-b089-a778d52c9ec3" containerName="registry-server" Jul 02 17:04:52.394225 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:52.394206 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lqrmq" Jul 02 17:04:52.396768 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:52.396745 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 02 17:04:52.396868 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:52.396749 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-s25vl\"" Jul 02 17:04:52.404985 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:52.404964 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lqrmq"] Jul 02 17:04:52.490316 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:52.490285 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcldp\" (UniqueName: \"kubernetes.io/projected/361c3f42-9644-402f-bd99-97d08fe0ff53-kube-api-access-rcldp\") pod \"dns-operator-controller-manager-864dd78c4c-lqrmq\" (UID: \"361c3f42-9644-402f-bd99-97d08fe0ff53\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lqrmq" Jul 02 17:04:52.591720 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:52.591690 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcldp\" (UniqueName: \"kubernetes.io/projected/361c3f42-9644-402f-bd99-97d08fe0ff53-kube-api-access-rcldp\") pod \"dns-operator-controller-manager-864dd78c4c-lqrmq\" (UID: \"361c3f42-9644-402f-bd99-97d08fe0ff53\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lqrmq" Jul 02 17:04:52.600888 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:52.600858 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcldp\" (UniqueName: \"kubernetes.io/projected/361c3f42-9644-402f-bd99-97d08fe0ff53-kube-api-access-rcldp\") pod \"dns-operator-controller-manager-864dd78c4c-lqrmq\" (UID: \"361c3f42-9644-402f-bd99-97d08fe0ff53\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lqrmq" Jul 02 17:04:52.705087 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:52.704995 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lqrmq" Jul 02 17:04:52.834749 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:52.834723 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lqrmq"] Jul 02 17:04:52.836488 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:04:52.836452 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod361c3f42_9644_402f_bd99_97d08fe0ff53.slice/crio-e532297ef98380507a18bfa505dfc8ca1d0727576c39e63c2c1549832e5f9c2c WatchSource:0}: Error finding container e532297ef98380507a18bfa505dfc8ca1d0727576c39e63c2c1549832e5f9c2c: Status 404 returned error can't find the container with id e532297ef98380507a18bfa505dfc8ca1d0727576c39e63c2c1549832e5f9c2c Jul 02 17:04:53.599283 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:53.599244 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lqrmq" event={"ID":"361c3f42-9644-402f-bd99-97d08fe0ff53","Type":"ContainerStarted","Data":"e532297ef98380507a18bfa505dfc8ca1d0727576c39e63c2c1549832e5f9c2c"} Jul 02 17:04:55.609503 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:55.609472 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lqrmq" event={"ID":"361c3f42-9644-402f-bd99-97d08fe0ff53","Type":"ContainerStarted","Data":"85607be1a3fc0d0017700271a8f5814549d7696038ed82b79f9ddcecc439af23"} Jul 02 17:04:55.609861 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:55.609583 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lqrmq" Jul 02 17:04:55.625493 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:55.625448 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lqrmq" podStartSLOduration=1.400296875 podStartE2EDuration="3.625435566s" totalImagesPullingTime="2.225138691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:04:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:04:55.624630722 +0000 UTC m=+628.937504368" watchObservedRunningTime="2026-07-02 17:04:55.625435566 +0000 UTC m=+628.938309211" Jul 02 17:04:55.858683 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:55.858646 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb"] Jul 02 17:04:55.862645 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:55.862592 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" Jul 02 17:04:55.864992 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:55.864976 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-g7lkl\"" Jul 02 17:04:55.869724 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:55.869690 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb"] Jul 02 17:04:55.923579 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:55.923553 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj5sm\" (UniqueName: \"kubernetes.io/projected/918df9a5-6c46-4d14-b4ef-bb046a7afafb-kube-api-access-gj5sm\") pod \"limitador-operator-controller-manager-859d7b78bd-xpbsb\" (UID: \"918df9a5-6c46-4d14-b4ef-bb046a7afafb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" Jul 02 17:04:56.023992 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:56.023956 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj5sm\" (UniqueName: \"kubernetes.io/projected/918df9a5-6c46-4d14-b4ef-bb046a7afafb-kube-api-access-gj5sm\") pod \"limitador-operator-controller-manager-859d7b78bd-xpbsb\" (UID: \"918df9a5-6c46-4d14-b4ef-bb046a7afafb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" Jul 02 17:04:56.038465 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:56.038436 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj5sm\" (UniqueName: \"kubernetes.io/projected/918df9a5-6c46-4d14-b4ef-bb046a7afafb-kube-api-access-gj5sm\") pod \"limitador-operator-controller-manager-859d7b78bd-xpbsb\" (UID: \"918df9a5-6c46-4d14-b4ef-bb046a7afafb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" Jul 02 17:04:56.172810 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:56.172730 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" Jul 02 17:04:56.304313 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:56.304291 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb"] Jul 02 17:04:56.306040 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:04:56.306012 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod918df9a5_6c46_4d14_b4ef_bb046a7afafb.slice/crio-203a62437fe0029e46ec9c666e91cef1ff42df556b5239fbcc2365d0078febf8 WatchSource:0}: Error finding container 203a62437fe0029e46ec9c666e91cef1ff42df556b5239fbcc2365d0078febf8: Status 404 returned error can't find the container with id 203a62437fe0029e46ec9c666e91cef1ff42df556b5239fbcc2365d0078febf8 Jul 02 17:04:56.615707 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:56.615675 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" event={"ID":"918df9a5-6c46-4d14-b4ef-bb046a7afafb","Type":"ContainerStarted","Data":"203a62437fe0029e46ec9c666e91cef1ff42df556b5239fbcc2365d0078febf8"} Jul 02 17:04:58.629128 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:58.629094 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" event={"ID":"918df9a5-6c46-4d14-b4ef-bb046a7afafb","Type":"ContainerStarted","Data":"28463f3871f2bd1e8659f2459454c54636360baa754435c181052c13666d6518"} Jul 02 17:04:58.629568 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:58.629294 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" Jul 02 17:04:58.646477 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:58.646430 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" podStartSLOduration=1.5513917369999999 podStartE2EDuration="3.646415539s" totalImagesPullingTime="2.095023802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:04:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:04:58.645523399 +0000 UTC m=+631.958397046" watchObservedRunningTime="2026-07-02 17:04:58.646415539 +0000 UTC m=+631.959289185" Jul 02 17:04:59.849745 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:59.849708 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-n7b24"] Jul 02 17:04:59.852357 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:59.852334 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-n7b24" Jul 02 17:04:59.855810 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:59.855789 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-shkq4\"" Jul 02 17:04:59.866930 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:59.866904 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-n7b24"] Jul 02 17:04:59.956925 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:04:59.956891 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgpst\" (UniqueName: \"kubernetes.io/projected/b650dd65-4cfa-48ac-adca-2cac5862b80c-kube-api-access-jgpst\") pod \"authorino-operator-75cc8d5cb7-n7b24\" (UID: \"b650dd65-4cfa-48ac-adca-2cac5862b80c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-n7b24" Jul 02 17:05:00.057746 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:00.057687 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgpst\" (UniqueName: \"kubernetes.io/projected/b650dd65-4cfa-48ac-adca-2cac5862b80c-kube-api-access-jgpst\") pod \"authorino-operator-75cc8d5cb7-n7b24\" (UID: \"b650dd65-4cfa-48ac-adca-2cac5862b80c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-n7b24" Jul 02 17:05:00.066998 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:00.066964 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgpst\" (UniqueName: \"kubernetes.io/projected/b650dd65-4cfa-48ac-adca-2cac5862b80c-kube-api-access-jgpst\") pod \"authorino-operator-75cc8d5cb7-n7b24\" (UID: \"b650dd65-4cfa-48ac-adca-2cac5862b80c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-n7b24" Jul 02 17:05:00.162301 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:00.162208 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-n7b24" Jul 02 17:05:00.299361 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:00.299336 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-n7b24"] Jul 02 17:05:00.301388 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:05:00.301359 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb650dd65_4cfa_48ac_adca_2cac5862b80c.slice/crio-11fc26e974e62231706e4b403aeca759663cff19bb24215c89a41a2fbe9f1752 WatchSource:0}: Error finding container 11fc26e974e62231706e4b403aeca759663cff19bb24215c89a41a2fbe9f1752: Status 404 returned error can't find the container with id 11fc26e974e62231706e4b403aeca759663cff19bb24215c89a41a2fbe9f1752 Jul 02 17:05:00.640778 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:00.640745 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-n7b24" event={"ID":"b650dd65-4cfa-48ac-adca-2cac5862b80c","Type":"ContainerStarted","Data":"11fc26e974e62231706e4b403aeca759663cff19bb24215c89a41a2fbe9f1752"} Jul 02 17:05:03.659284 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:03.659249 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-n7b24" event={"ID":"b650dd65-4cfa-48ac-adca-2cac5862b80c","Type":"ContainerStarted","Data":"bac2b90e30bbe4a0ea1aae6f32e5ac9b71660d64a56bcf0713c55181dbfd658f"} Jul 02 17:05:03.659700 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:03.659326 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-n7b24" Jul 02 17:05:03.677440 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:03.677392 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-n7b24" podStartSLOduration=2.099289776 podStartE2EDuration="4.677362093s" totalImagesPullingTime="2.578072317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:05:03.676372225 +0000 UTC m=+636.989245871" watchObservedRunningTime="2026-07-02 17:05:03.677362093 +0000 UTC m=+636.990235740" Jul 02 17:05:04.408768 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:04.408725 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc"] Jul 02 17:05:04.412054 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:04.412038 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:04.415896 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:04.415877 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6frrn\"" Jul 02 17:05:04.438241 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:04.438218 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc"] Jul 02 17:05:04.497528 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:04.497500 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da878e2e-bb57-4566-85f5-511adf69d153-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4bmrc\" (UID: \"da878e2e-bb57-4566-85f5-511adf69d153\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:04.497661 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:04.497540 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr6ph\" (UniqueName: \"kubernetes.io/projected/da878e2e-bb57-4566-85f5-511adf69d153-kube-api-access-jr6ph\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4bmrc\" (UID: \"da878e2e-bb57-4566-85f5-511adf69d153\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:04.598434 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:04.598391 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da878e2e-bb57-4566-85f5-511adf69d153-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4bmrc\" (UID: \"da878e2e-bb57-4566-85f5-511adf69d153\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:04.598570 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:04.598447 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr6ph\" (UniqueName: \"kubernetes.io/projected/da878e2e-bb57-4566-85f5-511adf69d153-kube-api-access-jr6ph\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4bmrc\" (UID: \"da878e2e-bb57-4566-85f5-511adf69d153\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:04.598777 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:04.598759 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da878e2e-bb57-4566-85f5-511adf69d153-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4bmrc\" (UID: \"da878e2e-bb57-4566-85f5-511adf69d153\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:04.606819 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:04.606797 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr6ph\" (UniqueName: \"kubernetes.io/projected/da878e2e-bb57-4566-85f5-511adf69d153-kube-api-access-jr6ph\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4bmrc\" (UID: \"da878e2e-bb57-4566-85f5-511adf69d153\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:04.721936 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:04.721859 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:04.865659 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:04.865634 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc"] Jul 02 17:05:04.867965 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:05:04.867940 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda878e2e_bb57_4566_85f5_511adf69d153.slice/crio-c5ee5caadaa7c0b52835c08082673836010e5ab93bc396a55d4f303760053cf5 WatchSource:0}: Error finding container c5ee5caadaa7c0b52835c08082673836010e5ab93bc396a55d4f303760053cf5: Status 404 returned error can't find the container with id c5ee5caadaa7c0b52835c08082673836010e5ab93bc396a55d4f303760053cf5 Jul 02 17:05:05.670001 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:05.669970 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" event={"ID":"da878e2e-bb57-4566-85f5-511adf69d153","Type":"ContainerStarted","Data":"c5ee5caadaa7c0b52835c08082673836010e5ab93bc396a55d4f303760053cf5"} Jul 02 17:05:06.617686 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:06.617659 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lqrmq" Jul 02 17:05:09.636349 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:09.636320 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" Jul 02 17:05:13.713330 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:13.713256 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" event={"ID":"da878e2e-bb57-4566-85f5-511adf69d153","Type":"ContainerStarted","Data":"5d478a6a0da7647b1a0e316929f0ccad04face2c4013b99f659133e9682486e8"} Jul 02 17:05:13.713658 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:13.713383 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:13.734637 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:13.734592 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" podStartSLOduration=1.339752464 podStartE2EDuration="9.734577996s" totalImagesPullingTime="8.394825532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:05:13.732364727 +0000 UTC m=+647.045238375" watchObservedRunningTime="2026-07-02 17:05:13.734577996 +0000 UTC m=+647.047451710" Jul 02 17:05:14.665801 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:14.665775 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-n7b24" Jul 02 17:05:24.721886 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:24.721857 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:26.533229 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.533196 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc"] Jul 02 17:05:26.533708 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.533414 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" podUID="da878e2e-bb57-4566-85f5-511adf69d153" containerName="manager" containerID="cri-o://5d478a6a0da7647b1a0e316929f0ccad04face2c4013b99f659133e9682486e8" gracePeriod=2 Jul 02 17:05:26.543303 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.543277 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc"] Jul 02 17:05:26.548894 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:05:26.548839 2556 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"da878e2e-bb57-4566-85f5-511adf69d153\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-4bmrc\": pods \"kuadrant-operator-controller-manager-8684c9ff49-4bmrc\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:26.554947 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.554920 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb"] Jul 02 17:05:26.555211 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.555182 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" podUID="918df9a5-6c46-4d14-b4ef-bb046a7afafb" containerName="manager" containerID="cri-o://28463f3871f2bd1e8659f2459454c54636360baa754435c181052c13666d6518" gracePeriod=2 Jul 02 17:05:26.559335 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:05:26.559043 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4bmrc\" is forbidden: User \"system:node:ip-10-0-129-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-102.ec2.internal' and this object" podUID="da878e2e-bb57-4566-85f5-511adf69d153" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:26.566641 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.565926 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b"] Jul 02 17:05:26.570304 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.568132 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da878e2e-bb57-4566-85f5-511adf69d153" containerName="manager" Jul 02 17:05:26.570304 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.569123 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="da878e2e-bb57-4566-85f5-511adf69d153" containerName="manager" Jul 02 17:05:26.570304 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.569371 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="da878e2e-bb57-4566-85f5-511adf69d153" containerName="manager" Jul 02 17:05:26.573849 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.573815 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb"] Jul 02 17:05:26.573999 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.573948 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" Jul 02 17:05:26.577473 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:05:26.577437 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4bmrc\" is forbidden: User \"system:node:ip-10-0-129-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-102.ec2.internal' and this object" podUID="da878e2e-bb57-4566-85f5-511adf69d153" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:26.587717 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.587679 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b"] Jul 02 17:05:26.592350 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.592310 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-42dl9"] Jul 02 17:05:26.592776 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.592758 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="918df9a5-6c46-4d14-b4ef-bb046a7afafb" containerName="manager" Jul 02 17:05:26.592882 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.592779 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="918df9a5-6c46-4d14-b4ef-bb046a7afafb" containerName="manager" Jul 02 17:05:26.592937 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.592897 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="918df9a5-6c46-4d14-b4ef-bb046a7afafb" containerName="manager" Jul 02 17:05:26.596062 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.596036 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-42dl9" Jul 02 17:05:26.605317 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.605285 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-42dl9"] Jul 02 17:05:26.608713 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:05:26.608671 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xpbsb\" is forbidden: User \"system:node:ip-10-0-129-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-102.ec2.internal' and this object" podUID="918df9a5-6c46-4d14-b4ef-bb046a7afafb" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" Jul 02 17:05:26.611312 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:05:26.611274 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xpbsb\" is forbidden: User \"system:node:ip-10-0-129-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-102.ec2.internal' and this object" podUID="918df9a5-6c46-4d14-b4ef-bb046a7afafb" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" Jul 02 17:05:26.632732 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:05:26.632695 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4bmrc\" is forbidden: User \"system:node:ip-10-0-129-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-102.ec2.internal' and this object" podUID="da878e2e-bb57-4566-85f5-511adf69d153" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:26.682862 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.682618 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6v7j\" (UniqueName: \"kubernetes.io/projected/7f0d41b8-6127-490a-a773-2302bb38c401-kube-api-access-d6v7j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h875b\" (UID: \"7f0d41b8-6127-490a-a773-2302bb38c401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" Jul 02 17:05:26.682862 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.682736 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f0d41b8-6127-490a-a773-2302bb38c401-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h875b\" (UID: \"7f0d41b8-6127-490a-a773-2302bb38c401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" Jul 02 17:05:26.682862 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.682768 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng2sx\" (UniqueName: \"kubernetes.io/projected/dd47754e-cfcc-4858-abb4-de54dc86fdef-kube-api-access-ng2sx\") pod \"limitador-operator-controller-manager-859d7b78bd-42dl9\" (UID: \"dd47754e-cfcc-4858-abb4-de54dc86fdef\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-42dl9" Jul 02 17:05:26.783464 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.783386 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6v7j\" (UniqueName: \"kubernetes.io/projected/7f0d41b8-6127-490a-a773-2302bb38c401-kube-api-access-d6v7j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h875b\" (UID: \"7f0d41b8-6127-490a-a773-2302bb38c401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" Jul 02 17:05:26.783625 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.783494 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f0d41b8-6127-490a-a773-2302bb38c401-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h875b\" (UID: \"7f0d41b8-6127-490a-a773-2302bb38c401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" Jul 02 17:05:26.783625 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.783548 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng2sx\" (UniqueName: \"kubernetes.io/projected/dd47754e-cfcc-4858-abb4-de54dc86fdef-kube-api-access-ng2sx\") pod \"limitador-operator-controller-manager-859d7b78bd-42dl9\" (UID: \"dd47754e-cfcc-4858-abb4-de54dc86fdef\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-42dl9" Jul 02 17:05:26.783625 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.783614 2556 generic.go:356] "Generic (PLEG): container finished" podID="918df9a5-6c46-4d14-b4ef-bb046a7afafb" containerID="28463f3871f2bd1e8659f2459454c54636360baa754435c181052c13666d6518" exitCode=0 Jul 02 17:05:26.783845 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.783825 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f0d41b8-6127-490a-a773-2302bb38c401-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h875b\" (UID: \"7f0d41b8-6127-490a-a773-2302bb38c401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" Jul 02 17:05:26.785838 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.785815 2556 generic.go:356] "Generic (PLEG): container finished" podID="da878e2e-bb57-4566-85f5-511adf69d153" containerID="5d478a6a0da7647b1a0e316929f0ccad04face2c4013b99f659133e9682486e8" exitCode=0 Jul 02 17:05:26.794589 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.794565 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6v7j\" (UniqueName: \"kubernetes.io/projected/7f0d41b8-6127-490a-a773-2302bb38c401-kube-api-access-d6v7j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h875b\" (UID: \"7f0d41b8-6127-490a-a773-2302bb38c401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" Jul 02 17:05:26.794781 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.794759 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng2sx\" (UniqueName: \"kubernetes.io/projected/dd47754e-cfcc-4858-abb4-de54dc86fdef-kube-api-access-ng2sx\") pod \"limitador-operator-controller-manager-859d7b78bd-42dl9\" (UID: \"dd47754e-cfcc-4858-abb4-de54dc86fdef\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-42dl9" Jul 02 17:05:26.828670 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.828650 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" Jul 02 17:05:26.830915 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.830899 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:26.831014 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:05:26.830994 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4bmrc\" is forbidden: User \"system:node:ip-10-0-129-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-102.ec2.internal' and this object" podUID="da878e2e-bb57-4566-85f5-511adf69d153" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:26.833068 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:05:26.833048 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xpbsb\" is forbidden: User \"system:node:ip-10-0-129-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-102.ec2.internal' and this object" podUID="918df9a5-6c46-4d14-b4ef-bb046a7afafb" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" Jul 02 17:05:26.834921 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:05:26.834904 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4bmrc\" is forbidden: User \"system:node:ip-10-0-129-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-102.ec2.internal' and this object" podUID="da878e2e-bb57-4566-85f5-511adf69d153" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:26.836920 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:05:26.836904 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xpbsb\" is forbidden: User \"system:node:ip-10-0-129-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-102.ec2.internal' and this object" podUID="918df9a5-6c46-4d14-b4ef-bb046a7afafb" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" Jul 02 17:05:26.981082 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.981048 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" Jul 02 17:05:26.985057 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.985041 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jr6ph\" (UniqueName: \"kubernetes.io/projected/da878e2e-bb57-4566-85f5-511adf69d153-kube-api-access-jr6ph\") pod \"da878e2e-bb57-4566-85f5-511adf69d153\" (UID: \"da878e2e-bb57-4566-85f5-511adf69d153\") " Jul 02 17:05:26.985114 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.985078 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gj5sm\" (UniqueName: \"kubernetes.io/projected/918df9a5-6c46-4d14-b4ef-bb046a7afafb-kube-api-access-gj5sm\") pod \"918df9a5-6c46-4d14-b4ef-bb046a7afafb\" (UID: \"918df9a5-6c46-4d14-b4ef-bb046a7afafb\") " Jul 02 17:05:26.985151 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.985116 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da878e2e-bb57-4566-85f5-511adf69d153-extensions-socket-volume\") pod \"da878e2e-bb57-4566-85f5-511adf69d153\" (UID: \"da878e2e-bb57-4566-85f5-511adf69d153\") " Jul 02 17:05:26.985677 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.985651 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da878e2e-bb57-4566-85f5-511adf69d153-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "da878e2e-bb57-4566-85f5-511adf69d153" (UID: "da878e2e-bb57-4566-85f5-511adf69d153"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:05:26.987289 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.987268 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/918df9a5-6c46-4d14-b4ef-bb046a7afafb-kube-api-access-gj5sm" (OuterVolumeSpecName: "kube-api-access-gj5sm") pod "918df9a5-6c46-4d14-b4ef-bb046a7afafb" (UID: "918df9a5-6c46-4d14-b4ef-bb046a7afafb"). InnerVolumeSpecName "kube-api-access-gj5sm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:05:26.987368 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.987290 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da878e2e-bb57-4566-85f5-511adf69d153-kube-api-access-jr6ph" (OuterVolumeSpecName: "kube-api-access-jr6ph") pod "da878e2e-bb57-4566-85f5-511adf69d153" (UID: "da878e2e-bb57-4566-85f5-511adf69d153"). InnerVolumeSpecName "kube-api-access-jr6ph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:05:26.988367 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:26.988349 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-42dl9" Jul 02 17:05:27.086804 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.086775 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jr6ph\" (UniqueName: \"kubernetes.io/projected/da878e2e-bb57-4566-85f5-511adf69d153-kube-api-access-jr6ph\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:05:27.086923 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.086809 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gj5sm\" (UniqueName: \"kubernetes.io/projected/918df9a5-6c46-4d14-b4ef-bb046a7afafb-kube-api-access-gj5sm\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:05:27.086923 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.086824 2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da878e2e-bb57-4566-85f5-511adf69d153-extensions-socket-volume\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:05:27.127047 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.127010 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b"] Jul 02 17:05:27.128394 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:05:27.128368 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f0d41b8_6127_490a_a773_2302bb38c401.slice/crio-ecd5429bd2d6bfaabe3483c9360f7947aa8e1506f99ff67417b19b52f72f481c WatchSource:0}: Error finding container ecd5429bd2d6bfaabe3483c9360f7947aa8e1506f99ff67417b19b52f72f481c: Status 404 returned error can't find the container with id ecd5429bd2d6bfaabe3483c9360f7947aa8e1506f99ff67417b19b52f72f481c Jul 02 17:05:27.144836 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.144812 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-42dl9"] Jul 02 17:05:27.147043 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:05:27.147021 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd47754e_cfcc_4858_abb4_de54dc86fdef.slice/crio-95774d039c25707f6eeb6ba0b6ccd66708878c374134edc8073f0dc05a7be286 WatchSource:0}: Error finding container 95774d039c25707f6eeb6ba0b6ccd66708878c374134edc8073f0dc05a7be286: Status 404 returned error can't find the container with id 95774d039c25707f6eeb6ba0b6ccd66708878c374134edc8073f0dc05a7be286 Jul 02 17:05:27.188783 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:05:27.188753 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xpbsb\" is forbidden: User \"system:node:ip-10-0-129-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-102.ec2.internal' and this object" podUID="918df9a5-6c46-4d14-b4ef-bb046a7afafb" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" Jul 02 17:05:27.191023 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:05:27.190987 2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4bmrc\" is forbidden: User \"system:node:ip-10-0-129-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-102.ec2.internal' and this object" podUID="da878e2e-bb57-4566-85f5-511adf69d153" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:27.191539 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.191520 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="918df9a5-6c46-4d14-b4ef-bb046a7afafb" path="/var/lib/kubelet/pods/918df9a5-6c46-4d14-b4ef-bb046a7afafb/volumes" Jul 02 17:05:27.191902 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.191887 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da878e2e-bb57-4566-85f5-511adf69d153" path="/var/lib/kubelet/pods/da878e2e-bb57-4566-85f5-511adf69d153/volumes" Jul 02 17:05:27.792425 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.792389 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-42dl9" event={"ID":"dd47754e-cfcc-4858-abb4-de54dc86fdef","Type":"ContainerStarted","Data":"beaed7ad2313eeaf9fc875ea2fa224e8ad9f9ed1155ff94df4a07dd4d0ef0c17"} Jul 02 17:05:27.792861 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.792451 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-42dl9" event={"ID":"dd47754e-cfcc-4858-abb4-de54dc86fdef","Type":"ContainerStarted","Data":"95774d039c25707f6eeb6ba0b6ccd66708878c374134edc8073f0dc05a7be286"} Jul 02 17:05:27.792861 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.792483 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-42dl9" Jul 02 17:05:27.794094 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.794066 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" event={"ID":"7f0d41b8-6127-490a-a773-2302bb38c401","Type":"ContainerStarted","Data":"a09b5f7a4ab2e2df9e67edb538d8bced05584102241cf7ba17df50199b4f48e7"} Jul 02 17:05:27.794238 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.794119 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" event={"ID":"7f0d41b8-6127-490a-a773-2302bb38c401","Type":"ContainerStarted","Data":"ecd5429bd2d6bfaabe3483c9360f7947aa8e1506f99ff67417b19b52f72f481c"} Jul 02 17:05:27.794238 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.794148 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" Jul 02 17:05:27.795391 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.795373 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xpbsb" Jul 02 17:05:27.795391 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.795384 2556 scope.go:117] "RemoveContainer" containerID="28463f3871f2bd1e8659f2459454c54636360baa754435c181052c13666d6518" Jul 02 17:05:27.796808 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.796785 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4bmrc" Jul 02 17:05:27.808463 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.808346 2556 scope.go:117] "RemoveContainer" containerID="5d478a6a0da7647b1a0e316929f0ccad04face2c4013b99f659133e9682486e8" Jul 02 17:05:27.812934 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.812896 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-42dl9" podStartSLOduration=1.812882729 podStartE2EDuration="1.812882729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:05:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:05:27.810487147 +0000 UTC m=+661.123360795" watchObservedRunningTime="2026-07-02 17:05:27.812882729 +0000 UTC m=+661.125756375" Jul 02 17:05:27.832869 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:27.832798 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" podStartSLOduration=1.832781678 podStartE2EDuration="1.832781678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:05:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:05:27.832270744 +0000 UTC m=+661.145144393" watchObservedRunningTime="2026-07-02 17:05:27.832781678 +0000 UTC m=+661.145655325" Jul 02 17:05:38.805959 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:38.805928 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-42dl9" Jul 02 17:05:38.806336 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:38.805997 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" Jul 02 17:05:42.064510 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.064473 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b"] Jul 02 17:05:42.064927 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.064770 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" podUID="7f0d41b8-6127-490a-a773-2302bb38c401" containerName="manager" containerID="cri-o://a09b5f7a4ab2e2df9e67edb538d8bced05584102241cf7ba17df50199b4f48e7" gracePeriod=10 Jul 02 17:05:42.269281 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.269239 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n"] Jul 02 17:05:42.272378 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.272359 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" Jul 02 17:05:42.292415 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.291811 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n"] Jul 02 17:05:42.312585 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.312554 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b15f237-e845-4a7d-8fa1-8bd7b4d170f8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ngx8n\" (UID: \"7b15f237-e845-4a7d-8fa1-8bd7b4d170f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" Jul 02 17:05:42.312699 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.312631 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8p5n\" (UniqueName: \"kubernetes.io/projected/7b15f237-e845-4a7d-8fa1-8bd7b4d170f8-kube-api-access-w8p5n\") pod \"kuadrant-operator-controller-manager-86878c769f-ngx8n\" (UID: \"7b15f237-e845-4a7d-8fa1-8bd7b4d170f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" Jul 02 17:05:42.320934 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.320877 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" Jul 02 17:05:42.413209 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.413142 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f0d41b8-6127-490a-a773-2302bb38c401-extensions-socket-volume\") pod \"7f0d41b8-6127-490a-a773-2302bb38c401\" (UID: \"7f0d41b8-6127-490a-a773-2302bb38c401\") " Jul 02 17:05:42.413380 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.413263 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6v7j\" (UniqueName: \"kubernetes.io/projected/7f0d41b8-6127-490a-a773-2302bb38c401-kube-api-access-d6v7j\") pod \"7f0d41b8-6127-490a-a773-2302bb38c401\" (UID: \"7f0d41b8-6127-490a-a773-2302bb38c401\") " Jul 02 17:05:42.413431 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.413395 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b15f237-e845-4a7d-8fa1-8bd7b4d170f8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ngx8n\" (UID: \"7b15f237-e845-4a7d-8fa1-8bd7b4d170f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" Jul 02 17:05:42.413534 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.413517 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8p5n\" (UniqueName: \"kubernetes.io/projected/7b15f237-e845-4a7d-8fa1-8bd7b4d170f8-kube-api-access-w8p5n\") pod \"kuadrant-operator-controller-manager-86878c769f-ngx8n\" (UID: \"7b15f237-e845-4a7d-8fa1-8bd7b4d170f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" Jul 02 17:05:42.413630 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.413609 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f0d41b8-6127-490a-a773-2302bb38c401-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7f0d41b8-6127-490a-a773-2302bb38c401" (UID: "7f0d41b8-6127-490a-a773-2302bb38c401"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:05:42.413822 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.413800 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b15f237-e845-4a7d-8fa1-8bd7b4d170f8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ngx8n\" (UID: \"7b15f237-e845-4a7d-8fa1-8bd7b4d170f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" Jul 02 17:05:42.415376 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.415351 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f0d41b8-6127-490a-a773-2302bb38c401-kube-api-access-d6v7j" (OuterVolumeSpecName: "kube-api-access-d6v7j") pod "7f0d41b8-6127-490a-a773-2302bb38c401" (UID: "7f0d41b8-6127-490a-a773-2302bb38c401"). InnerVolumeSpecName "kube-api-access-d6v7j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:05:42.426613 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.426588 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8p5n\" (UniqueName: \"kubernetes.io/projected/7b15f237-e845-4a7d-8fa1-8bd7b4d170f8-kube-api-access-w8p5n\") pod \"kuadrant-operator-controller-manager-86878c769f-ngx8n\" (UID: \"7b15f237-e845-4a7d-8fa1-8bd7b4d170f8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" Jul 02 17:05:42.514612 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.514575 2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f0d41b8-6127-490a-a773-2302bb38c401-extensions-socket-volume\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:05:42.514612 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.514611 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6v7j\" (UniqueName: \"kubernetes.io/projected/7f0d41b8-6127-490a-a773-2302bb38c401-kube-api-access-d6v7j\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:05:42.586429 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.586348 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" Jul 02 17:05:42.721310 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.721283 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n"] Jul 02 17:05:42.723294 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:05:42.723268 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b15f237_e845_4a7d_8fa1_8bd7b4d170f8.slice/crio-a148a2a716ce39253139ff69cd234db8a10281860c873b077d47b49e21c70c49 WatchSource:0}: Error finding container a148a2a716ce39253139ff69cd234db8a10281860c873b077d47b49e21c70c49: Status 404 returned error can't find the container with id a148a2a716ce39253139ff69cd234db8a10281860c873b077d47b49e21c70c49 Jul 02 17:05:42.876672 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.876583 2556 generic.go:356] "Generic (PLEG): container finished" podID="7f0d41b8-6127-490a-a773-2302bb38c401" containerID="a09b5f7a4ab2e2df9e67edb538d8bced05584102241cf7ba17df50199b4f48e7" exitCode=0 Jul 02 17:05:42.876672 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.876630 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" Jul 02 17:05:42.876870 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.876671 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" event={"ID":"7f0d41b8-6127-490a-a773-2302bb38c401","Type":"ContainerDied","Data":"a09b5f7a4ab2e2df9e67edb538d8bced05584102241cf7ba17df50199b4f48e7"} Jul 02 17:05:42.876870 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.876728 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b" event={"ID":"7f0d41b8-6127-490a-a773-2302bb38c401","Type":"ContainerDied","Data":"ecd5429bd2d6bfaabe3483c9360f7947aa8e1506f99ff67417b19b52f72f481c"} Jul 02 17:05:42.876870 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.876757 2556 scope.go:117] "RemoveContainer" containerID="a09b5f7a4ab2e2df9e67edb538d8bced05584102241cf7ba17df50199b4f48e7" Jul 02 17:05:42.878499 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.878473 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" event={"ID":"7b15f237-e845-4a7d-8fa1-8bd7b4d170f8","Type":"ContainerStarted","Data":"a2584e3c6699d08e10d398ae1f6e9cd9bccec18a0596d005fd076a0c3d3e7644"} Jul 02 17:05:42.878661 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.878517 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" event={"ID":"7b15f237-e845-4a7d-8fa1-8bd7b4d170f8","Type":"ContainerStarted","Data":"a148a2a716ce39253139ff69cd234db8a10281860c873b077d47b49e21c70c49"} Jul 02 17:05:42.878661 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.878601 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" Jul 02 17:05:42.886828 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.886758 2556 scope.go:117] "RemoveContainer" containerID="a09b5f7a4ab2e2df9e67edb538d8bced05584102241cf7ba17df50199b4f48e7" Jul 02 17:05:42.887055 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:05:42.887037 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a09b5f7a4ab2e2df9e67edb538d8bced05584102241cf7ba17df50199b4f48e7\": container with ID starting with a09b5f7a4ab2e2df9e67edb538d8bced05584102241cf7ba17df50199b4f48e7 not found: ID does not exist" containerID="a09b5f7a4ab2e2df9e67edb538d8bced05584102241cf7ba17df50199b4f48e7" Jul 02 17:05:42.887110 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.887064 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a09b5f7a4ab2e2df9e67edb538d8bced05584102241cf7ba17df50199b4f48e7"} err="failed to get container status \"a09b5f7a4ab2e2df9e67edb538d8bced05584102241cf7ba17df50199b4f48e7\": rpc error: code = NotFound desc = could not find container \"a09b5f7a4ab2e2df9e67edb538d8bced05584102241cf7ba17df50199b4f48e7\": container with ID starting with a09b5f7a4ab2e2df9e67edb538d8bced05584102241cf7ba17df50199b4f48e7 not found: ID does not exist" Jul 02 17:05:42.901610 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.901565 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" podStartSLOduration=0.901521317 podStartE2EDuration="901.521317ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:05:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:05:42.900220405 +0000 UTC m=+676.213094053" watchObservedRunningTime="2026-07-02 17:05:42.901521317 +0000 UTC m=+676.214394998" Jul 02 17:05:42.914140 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.914117 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b"] Jul 02 17:05:42.918377 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:42.918351 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h875b"] Jul 02 17:05:43.190117 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:43.190024 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f0d41b8-6127-490a-a773-2302bb38c401" path="/var/lib/kubelet/pods/7f0d41b8-6127-490a-a773-2302bb38c401/volumes" Jul 02 17:05:53.888553 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:53.888522 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" Jul 02 17:05:58.989729 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:58.989685 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf"] Jul 02 17:05:58.990285 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:58.990267 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f0d41b8-6127-490a-a773-2302bb38c401" containerName="manager" Jul 02 17:05:58.990361 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:58.990288 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f0d41b8-6127-490a-a773-2302bb38c401" containerName="manager" Jul 02 17:05:58.990431 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:58.990418 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f0d41b8-6127-490a-a773-2302bb38c401" containerName="manager" Jul 02 17:05:58.992694 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:58.992674 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:58.994966 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:58.994949 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-hgb9p\"" Jul 02 17:05:59.003673 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.003647 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf"] Jul 02 17:05:59.053911 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.053874 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6335ba4-07ee-4581-9d37-5d3755535085-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.054066 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.053922 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6335ba4-07ee-4581-9d37-5d3755535085-workload-certs\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.054066 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.053940 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6335ba4-07ee-4581-9d37-5d3755535085-istio-data\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.054066 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.054007 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6335ba4-07ee-4581-9d37-5d3755535085-workload-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.054066 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.054036 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6335ba4-07ee-4581-9d37-5d3755535085-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.054066 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.054065 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb6sk\" (UniqueName: \"kubernetes.io/projected/b6335ba4-07ee-4581-9d37-5d3755535085-kube-api-access-pb6sk\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.054276 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.054141 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6335ba4-07ee-4581-9d37-5d3755535085-istio-envoy\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.054276 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.054192 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6335ba4-07ee-4581-9d37-5d3755535085-credential-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.054276 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.054212 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6335ba4-07ee-4581-9d37-5d3755535085-istio-token\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.155060 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.155020 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6335ba4-07ee-4581-9d37-5d3755535085-istio-envoy\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.155060 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.155063 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6335ba4-07ee-4581-9d37-5d3755535085-credential-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.155355 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.155092 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6335ba4-07ee-4581-9d37-5d3755535085-istio-token\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.155355 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.155121 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6335ba4-07ee-4581-9d37-5d3755535085-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.155355 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.155191 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6335ba4-07ee-4581-9d37-5d3755535085-workload-certs\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.155355 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.155221 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6335ba4-07ee-4581-9d37-5d3755535085-istio-data\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.155355 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.155253 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6335ba4-07ee-4581-9d37-5d3755535085-workload-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.155355 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.155280 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6335ba4-07ee-4581-9d37-5d3755535085-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.155355 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.155317 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb6sk\" (UniqueName: \"kubernetes.io/projected/b6335ba4-07ee-4581-9d37-5d3755535085-kube-api-access-pb6sk\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.155706 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.155586 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6335ba4-07ee-4581-9d37-5d3755535085-credential-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.155706 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.155631 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6335ba4-07ee-4581-9d37-5d3755535085-workload-certs\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.155706 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.155696 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6335ba4-07ee-4581-9d37-5d3755535085-istio-data\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.155856 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.155806 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6335ba4-07ee-4581-9d37-5d3755535085-workload-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.156141 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.156119 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6335ba4-07ee-4581-9d37-5d3755535085-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.157396 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.157374 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6335ba4-07ee-4581-9d37-5d3755535085-istio-envoy\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.157545 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.157526 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6335ba4-07ee-4581-9d37-5d3755535085-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.162947 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.162926 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6335ba4-07ee-4581-9d37-5d3755535085-istio-token\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.163332 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.163310 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb6sk\" (UniqueName: \"kubernetes.io/projected/b6335ba4-07ee-4581-9d37-5d3755535085-kube-api-access-pb6sk\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-57xrf\" (UID: \"b6335ba4-07ee-4581-9d37-5d3755535085\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.303857 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.303821 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:05:59.440892 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.440867 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf"] Jul 02 17:05:59.443077 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:05:59.443050 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6335ba4_07ee_4581_9d37_5d3755535085.slice/crio-87cc193ac9fa32568e20aa9b4d5fbab12e150b5b70d06d20ff9040e6bd22d4e3 WatchSource:0}: Error finding container 87cc193ac9fa32568e20aa9b4d5fbab12e150b5b70d06d20ff9040e6bd22d4e3: Status 404 returned error can't find the container with id 87cc193ac9fa32568e20aa9b4d5fbab12e150b5b70d06d20ff9040e6bd22d4e3 Jul 02 17:05:59.445380 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.445346 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:05:59.445475 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.445421 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:05:59.445475 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.445451 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:05:59.968574 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.968538 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" event={"ID":"b6335ba4-07ee-4581-9d37-5d3755535085","Type":"ContainerStarted","Data":"38cfb6a14da88b250ae464ce8d4dc7b289f83649c747fd6efc37fa7923eb93f5"} Jul 02 17:05:59.968760 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.968599 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" event={"ID":"b6335ba4-07ee-4581-9d37-5d3755535085","Type":"ContainerStarted","Data":"87cc193ac9fa32568e20aa9b4d5fbab12e150b5b70d06d20ff9040e6bd22d4e3"} Jul 02 17:05:59.993487 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:05:59.993438 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" podStartSLOduration=1.9934198950000002 podStartE2EDuration="1.993419895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:05:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:05:59.991893491 +0000 UTC m=+693.304767138" watchObservedRunningTime="2026-07-02 17:05:59.993419895 +0000 UTC m=+693.306293543" Jul 02 17:06:00.304028 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:00.303990 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:06:00.308589 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:00.308564 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:06:00.974480 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:00.974444 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:06:00.975380 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:00.975360 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:06:10.754146 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:10.754105 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:06:10.757358 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:10.757334 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-gn5fl" Jul 02 17:06:10.759962 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:10.759940 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 02 17:06:10.760064 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:10.760046 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8c4pr\"" Jul 02 17:06:10.765635 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:10.765611 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:06:10.794700 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:10.794663 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:06:10.869404 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:10.869364 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdb87\" (UniqueName: \"kubernetes.io/projected/5d7f90bf-6317-42be-8e5b-a81e2a5f981c-kube-api-access-mdb87\") pod \"limitador-limitador-54d6d988d5-gn5fl\" (UID: \"5d7f90bf-6317-42be-8e5b-a81e2a5f981c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gn5fl" Jul 02 17:06:10.869561 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:10.869415 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5d7f90bf-6317-42be-8e5b-a81e2a5f981c-config-file\") pod \"limitador-limitador-54d6d988d5-gn5fl\" (UID: \"5d7f90bf-6317-42be-8e5b-a81e2a5f981c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gn5fl" Jul 02 17:06:10.970140 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:10.970109 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5d7f90bf-6317-42be-8e5b-a81e2a5f981c-config-file\") pod \"limitador-limitador-54d6d988d5-gn5fl\" (UID: \"5d7f90bf-6317-42be-8e5b-a81e2a5f981c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gn5fl" Jul 02 17:06:10.970313 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:10.970228 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdb87\" (UniqueName: \"kubernetes.io/projected/5d7f90bf-6317-42be-8e5b-a81e2a5f981c-kube-api-access-mdb87\") pod \"limitador-limitador-54d6d988d5-gn5fl\" (UID: \"5d7f90bf-6317-42be-8e5b-a81e2a5f981c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gn5fl" Jul 02 17:06:10.970756 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:10.970736 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5d7f90bf-6317-42be-8e5b-a81e2a5f981c-config-file\") pod \"limitador-limitador-54d6d988d5-gn5fl\" (UID: \"5d7f90bf-6317-42be-8e5b-a81e2a5f981c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gn5fl" Jul 02 17:06:10.978631 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:10.978608 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdb87\" (UniqueName: \"kubernetes.io/projected/5d7f90bf-6317-42be-8e5b-a81e2a5f981c-kube-api-access-mdb87\") pod \"limitador-limitador-54d6d988d5-gn5fl\" (UID: \"5d7f90bf-6317-42be-8e5b-a81e2a5f981c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gn5fl" Jul 02 17:06:11.070232 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:11.070121 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-gn5fl" Jul 02 17:06:11.196474 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:11.196453 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:06:11.198017 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:06:11.197986 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d7f90bf_6317_42be_8e5b_a81e2a5f981c.slice/crio-c900488e89b231375082cf1e4ba51d5f4fedfafb366257a3043960d80734ded3 WatchSource:0}: Error finding container c900488e89b231375082cf1e4ba51d5f4fedfafb366257a3043960d80734ded3: Status 404 returned error can't find the container with id c900488e89b231375082cf1e4ba51d5f4fedfafb366257a3043960d80734ded3 Jul 02 17:06:11.199906 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:11.199885 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 17:06:12.033816 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:12.033759 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-gn5fl" event={"ID":"5d7f90bf-6317-42be-8e5b-a81e2a5f981c","Type":"ContainerStarted","Data":"c900488e89b231375082cf1e4ba51d5f4fedfafb366257a3043960d80734ded3"} Jul 02 17:06:14.045995 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:14.045960 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-gn5fl" event={"ID":"5d7f90bf-6317-42be-8e5b-a81e2a5f981c","Type":"ContainerStarted","Data":"9a170bbf98e58746b01e3273568b3bbbcd9a8b6b75199665bca5203efdff7a07"} Jul 02 17:06:14.046480 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:14.046029 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-gn5fl" Jul 02 17:06:14.063382 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:14.063334 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-gn5fl" podStartSLOduration=1.669558213 podStartE2EDuration="4.063317062s" totalImagesPullingTime="2.393758849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:06:14.061690782 +0000 UTC m=+707.374564429" watchObservedRunningTime="2026-07-02 17:06:14.063317062 +0000 UTC m=+707.376190721" Jul 02 17:06:25.052414 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:06:25.052340 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-gn5fl" Jul 02 17:07:24.329618 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:24.329580 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:07:31.966182 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:31.966122 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5d7b8c9489-57xrf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": read tcp 10.133.0.2:41170->10.133.0.40:15021: read: connection reset by peer" start-of-body= Jul 02 17:07:31.966531 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:31.966227 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" podUID="b6335ba4-07ee-4581-9d37-5d3755535085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": read tcp 10.133.0.2:41170->10.133.0.40:15021: read: connection reset by peer" Jul 02 17:07:32.450651 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:32.450619 2556 generic.go:356] "Generic (PLEG): container finished" podID="b6335ba4-07ee-4581-9d37-5d3755535085" containerID="38cfb6a14da88b250ae464ce8d4dc7b289f83649c747fd6efc37fa7923eb93f5" exitCode=0 Jul 02 17:07:32.450821 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:32.450700 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" event={"ID":"b6335ba4-07ee-4581-9d37-5d3755535085","Type":"ContainerDied","Data":"38cfb6a14da88b250ae464ce8d4dc7b289f83649c747fd6efc37fa7923eb93f5"} Jul 02 17:07:32.451094 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:32.451078 2556 scope.go:117] "RemoveContainer" containerID="38cfb6a14da88b250ae464ce8d4dc7b289f83649c747fd6efc37fa7923eb93f5" Jul 02 17:07:32.452036 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:32.452009 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:07:32.452139 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:32.452088 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:07:32.452222 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:32.452135 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:07:33.457659 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:33.457624 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" event={"ID":"b6335ba4-07ee-4581-9d37-5d3755535085","Type":"ContainerStarted","Data":"5fa9b54b0ea52c7f30473b7c3cb5d0879b8561bcbfcbf045d9bc518a09fd040b"} Jul 02 17:07:34.305251 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:34.305213 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:07:34.305472 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:34.305264 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5d7b8c9489-57xrf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 02 17:07:34.305472 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:34.305335 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" podUID="b6335ba4-07ee-4581-9d37-5d3755535085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 02 17:07:35.304516 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:35.304481 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5d7b8c9489-57xrf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 02 17:07:35.305048 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:35.305018 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" podUID="b6335ba4-07ee-4581-9d37-5d3755535085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 02 17:07:36.304959 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:36.304920 2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5d7b8c9489-57xrf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 02 17:07:36.305592 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:36.305558 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" podUID="b6335ba4-07ee-4581-9d37-5d3755535085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 02 17:07:37.307892 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:37.307862 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:07:37.308274 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:37.308184 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:07:37.308818 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:07:37.308798 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-57xrf" Jul 02 17:08:04.502281 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:04.502201 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:08:11.656414 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:11.656380 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:08:24.371011 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:24.370974 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:08:50.865554 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:50.865520 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:08:54.485538 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.485506 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b"] Jul 02 17:08:54.488922 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.488904 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.492367 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.492347 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 02 17:08:54.493365 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.493346 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 02 17:08:54.493517 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.493502 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gxrr7\"" Jul 02 17:08:54.493568 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.493502 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 02 17:08:54.502606 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.502581 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b"] Jul 02 17:08:54.512129 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.512108 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rn5d\" (UniqueName: \"kubernetes.io/projected/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-kube-api-access-5rn5d\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.512241 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.512143 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.512241 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.512185 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.512334 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.512246 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.512334 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.512294 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.512334 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.512327 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.613704 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.613680 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rn5d\" (UniqueName: \"kubernetes.io/projected/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-kube-api-access-5rn5d\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.613831 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.613721 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.613831 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.613769 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.613831 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.613815 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.613996 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.613849 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.614142 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.614120 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.614348 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.614322 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.614502 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.614478 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.614825 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.614806 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.616574 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.616512 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.616839 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.616820 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.625927 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.625904 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rn5d\" (UniqueName: \"kubernetes.io/projected/6b4077ad-b97c-47a3-8d31-f8cb82e9a88a-kube-api-access-5rn5d\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dcq4b\" (UID: \"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.800317 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.800286 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:08:54.937278 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:54.937255 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b"] Jul 02 17:08:54.939311 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:08:54.939284 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b4077ad_b97c_47a3_8d31_f8cb82e9a88a.slice/crio-c996224082332fdbc065563832a7824e570014470f38779f72db0d4c728a769b WatchSource:0}: Error finding container c996224082332fdbc065563832a7824e570014470f38779f72db0d4c728a769b: Status 404 returned error can't find the container with id c996224082332fdbc065563832a7824e570014470f38779f72db0d4c728a769b Jul 02 17:08:55.862127 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:55.862083 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" event={"ID":"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a","Type":"ContainerStarted","Data":"c996224082332fdbc065563832a7824e570014470f38779f72db0d4c728a769b"} Jul 02 17:08:56.787186 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:56.787140 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:08:56.868686 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:56.868652 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" event={"ID":"6b4077ad-b97c-47a3-8d31-f8cb82e9a88a","Type":"ContainerStarted","Data":"cfd8f21e87cdfc6be002643063b0d2383bcc41488fa485cb0c2de0b4ac39c9cb"} Jul 02 17:08:56.898388 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:08:56.898333 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" podStartSLOduration=1.7898924090000001 podStartE2EDuration="2.898316845s" totalImagesPullingTime="1.108424436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:08:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:08:56.896393288 +0000 UTC m=+870.209266934" watchObservedRunningTime="2026-07-02 17:08:56.898316845 +0000 UTC m=+870.211190491" Jul 02 17:09:03.487898 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:09:03.487862 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:09:04.800714 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:09:04.800678 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:09:04.801206 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:09:04.800746 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:09:04.816198 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:09:04.816168 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:09:04.920510 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:09:04.920484 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dcq4b" Jul 02 17:09:27.181687 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:09:27.181604 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 17:09:27.184645 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:09:27.184623 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 17:10:16.857231 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:10:16.857197 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:10:27.263779 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:10:27.263749 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:10:36.660862 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:10:36.660827 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:10:44.266825 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:10:44.266785 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:10:53.158745 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:10:53.158661 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:11:01.254723 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:11:01.254688 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:12:02.977198 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:12:02.977147 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:12:13.462046 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:12:13.462012 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:12:20.758402 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:12:20.758316 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:12:31.353793 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:12:31.353759 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:12:40.657715 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:12:40.657682 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:12:51.664812 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:12:51.664777 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:13:01.575008 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:13:01.574967 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:13:10.969481 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:13:10.969445 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:13:33.778986 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:13:33.778944 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:13:43.571416 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:13:43.571380 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:14:06.368276 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:14:06.368192 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:14:18.773751 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:14:18.773711 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:14:27.220959 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:14:27.220930 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 17:14:27.221601 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:14:27.221583 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 17:14:32.863467 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:14:32.863433 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:14:42.271512 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:14:42.271477 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:14:51.599903 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:14:51.599865 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:15:01.570411 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:01.570374 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:15:10.563912 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:10.563870 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:15:37.766256 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:37.766222 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n"] Jul 02 17:15:37.766785 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:37.766458 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" podUID="7b15f237-e845-4a7d-8fa1-8bd7b4d170f8" containerName="manager" containerID="cri-o://a2584e3c6699d08e10d398ae1f6e9cd9bccec18a0596d005fd076a0c3d3e7644" gracePeriod=10 Jul 02 17:15:40.900820 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:40.900780 2556 generic.go:356] "Generic (PLEG): container finished" podID="7b15f237-e845-4a7d-8fa1-8bd7b4d170f8" containerID="a2584e3c6699d08e10d398ae1f6e9cd9bccec18a0596d005fd076a0c3d3e7644" exitCode=0 Jul 02 17:15:40.901185 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:40.900827 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" event={"ID":"7b15f237-e845-4a7d-8fa1-8bd7b4d170f8","Type":"ContainerDied","Data":"a2584e3c6699d08e10d398ae1f6e9cd9bccec18a0596d005fd076a0c3d3e7644"} Jul 02 17:15:40.928194 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:40.928147 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" Jul 02 17:15:40.980866 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:40.980792 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b15f237-e845-4a7d-8fa1-8bd7b4d170f8-extensions-socket-volume\") pod \"7b15f237-e845-4a7d-8fa1-8bd7b4d170f8\" (UID: \"7b15f237-e845-4a7d-8fa1-8bd7b4d170f8\") " Jul 02 17:15:40.980866 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:40.980841 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8p5n\" (UniqueName: \"kubernetes.io/projected/7b15f237-e845-4a7d-8fa1-8bd7b4d170f8-kube-api-access-w8p5n\") pod \"7b15f237-e845-4a7d-8fa1-8bd7b4d170f8\" (UID: \"7b15f237-e845-4a7d-8fa1-8bd7b4d170f8\") " Jul 02 17:15:40.981268 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:40.981236 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b15f237-e845-4a7d-8fa1-8bd7b4d170f8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7b15f237-e845-4a7d-8fa1-8bd7b4d170f8" (UID: "7b15f237-e845-4a7d-8fa1-8bd7b4d170f8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:15:40.982820 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:40.982798 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b15f237-e845-4a7d-8fa1-8bd7b4d170f8-kube-api-access-w8p5n" (OuterVolumeSpecName: "kube-api-access-w8p5n") pod "7b15f237-e845-4a7d-8fa1-8bd7b4d170f8" (UID: "7b15f237-e845-4a7d-8fa1-8bd7b4d170f8"). InnerVolumeSpecName "kube-api-access-w8p5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:15:41.081521 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:41.081494 2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b15f237-e845-4a7d-8fa1-8bd7b4d170f8-extensions-socket-volume\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:15:41.081521 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:41.081514 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8p5n\" (UniqueName: \"kubernetes.io/projected/7b15f237-e845-4a7d-8fa1-8bd7b4d170f8-kube-api-access-w8p5n\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:15:41.907001 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:41.906971 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" Jul 02 17:15:41.907001 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:41.906990 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n" event={"ID":"7b15f237-e845-4a7d-8fa1-8bd7b4d170f8","Type":"ContainerDied","Data":"a148a2a716ce39253139ff69cd234db8a10281860c873b077d47b49e21c70c49"} Jul 02 17:15:41.907501 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:41.907045 2556 scope.go:117] "RemoveContainer" containerID="a2584e3c6699d08e10d398ae1f6e9cd9bccec18a0596d005fd076a0c3d3e7644" Jul 02 17:15:41.925926 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:41.925892 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n"] Jul 02 17:15:41.930584 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:41.930546 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ngx8n"] Jul 02 17:15:43.190520 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:15:43.190487 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b15f237-e845-4a7d-8fa1-8bd7b4d170f8" path="/var/lib/kubelet/pods/7b15f237-e845-4a7d-8fa1-8bd7b4d170f8/volumes" Jul 02 17:16:38.923667 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:38.923632 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x9j96"] Jul 02 17:16:38.924062 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:38.924028 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b15f237-e845-4a7d-8fa1-8bd7b4d170f8" containerName="manager" Jul 02 17:16:38.924062 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:38.924038 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b15f237-e845-4a7d-8fa1-8bd7b4d170f8" containerName="manager" Jul 02 17:16:38.924134 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:38.924114 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b15f237-e845-4a7d-8fa1-8bd7b4d170f8" containerName="manager" Jul 02 17:16:38.928323 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:38.928305 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x9j96" Jul 02 17:16:38.931897 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:38.931879 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6frrn\"" Jul 02 17:16:38.946782 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:38.946755 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x9j96"] Jul 02 17:16:39.062190 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:39.062123 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0011b5c4-160c-4e50-aa00-9611104702e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x9j96\" (UID: \"0011b5c4-160c-4e50-aa00-9611104702e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x9j96" Jul 02 17:16:39.062190 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:39.062194 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zht26\" (UniqueName: \"kubernetes.io/projected/0011b5c4-160c-4e50-aa00-9611104702e2-kube-api-access-zht26\") pod \"kuadrant-operator-controller-manager-86878c769f-x9j96\" (UID: \"0011b5c4-160c-4e50-aa00-9611104702e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x9j96" Jul 02 17:16:39.163521 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:39.163480 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0011b5c4-160c-4e50-aa00-9611104702e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x9j96\" (UID: \"0011b5c4-160c-4e50-aa00-9611104702e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x9j96" Jul 02 17:16:39.163521 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:39.163527 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zht26\" (UniqueName: \"kubernetes.io/projected/0011b5c4-160c-4e50-aa00-9611104702e2-kube-api-access-zht26\") pod \"kuadrant-operator-controller-manager-86878c769f-x9j96\" (UID: \"0011b5c4-160c-4e50-aa00-9611104702e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x9j96" Jul 02 17:16:39.163951 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:39.163928 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0011b5c4-160c-4e50-aa00-9611104702e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x9j96\" (UID: \"0011b5c4-160c-4e50-aa00-9611104702e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x9j96" Jul 02 17:16:39.180080 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:39.180008 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zht26\" (UniqueName: \"kubernetes.io/projected/0011b5c4-160c-4e50-aa00-9611104702e2-kube-api-access-zht26\") pod \"kuadrant-operator-controller-manager-86878c769f-x9j96\" (UID: \"0011b5c4-160c-4e50-aa00-9611104702e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x9j96" Jul 02 17:16:39.238697 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:39.238656 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x9j96" Jul 02 17:16:39.386104 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:39.386078 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x9j96"] Jul 02 17:16:39.388262 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:16:39.388234 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0011b5c4_160c_4e50_aa00_9611104702e2.slice/crio-21e592f344706d4d20a45f2f4f199a07ad35d8f59333a316b0e41073bc4fe952 WatchSource:0}: Error finding container 21e592f344706d4d20a45f2f4f199a07ad35d8f59333a316b0e41073bc4fe952: Status 404 returned error can't find the container with id 21e592f344706d4d20a45f2f4f199a07ad35d8f59333a316b0e41073bc4fe952 Jul 02 17:16:40.197746 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:40.197712 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x9j96" event={"ID":"0011b5c4-160c-4e50-aa00-9611104702e2","Type":"ContainerStarted","Data":"70972d6a5f7f2434543970269fd3d2700ae1b85369924ec4964aa6ac1ffc95e6"} Jul 02 17:16:40.198210 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:40.197762 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x9j96" event={"ID":"0011b5c4-160c-4e50-aa00-9611104702e2","Type":"ContainerStarted","Data":"21e592f344706d4d20a45f2f4f199a07ad35d8f59333a316b0e41073bc4fe952"} Jul 02 17:16:40.198210 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:40.197851 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x9j96" Jul 02 17:16:40.222725 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:40.222536 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x9j96" podStartSLOduration=2.222516953 podStartE2EDuration="2.222516953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:16:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:16:40.22067668 +0000 UTC m=+1333.533550328" watchObservedRunningTime="2026-07-02 17:16:40.222516953 +0000 UTC m=+1333.535390602" Jul 02 17:16:51.204813 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:51.204739 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x9j96" Jul 02 17:16:58.959039 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:16:58.958997 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:17:10.175455 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:17:10.175420 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:17:22.865104 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:17:22.865065 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:17:33.964740 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:17:33.964704 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:17:45.070123 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:17:45.070089 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:17:55.458906 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:17:55.458870 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:18:04.663829 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:18:04.663784 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:18:15.064799 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:18:15.064764 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:18:26.165146 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:18:26.165063 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:18:34.062341 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:18:34.062306 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:18:42.563752 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:18:42.563718 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:18:49.162711 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:18:49.162673 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:19:10.473106 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:19:10.473066 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:19:19.466251 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:19:19.466200 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:19:27.259342 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:19:27.259317 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 17:19:27.260982 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:19:27.260960 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 17:19:30.374050 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:19:30.374016 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:19:39.661563 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:19:39.661521 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:19:55.268448 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:19:55.268358 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:20:06.569342 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:20:06.569308 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:20:15.565674 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:20:15.565637 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:20:25.771206 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:20:25.771174 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:20:34.863743 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:20:34.863705 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:20:43.965103 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:20:43.965061 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:20:57.581376 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:20:57.581342 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:21:06.504298 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:21:06.504254 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:21:15.972613 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:21:15.972578 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:21:27.271216 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:21:27.271109 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:21:46.468101 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:21:46.468063 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:21:57.571727 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:21:57.571692 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:22:09.266729 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:09.266694 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:22:21.165256 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:21.165220 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:22:33.518407 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:33.518369 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:22:47.868680 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:47.868639 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:22:53.505376 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.505344 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw"] Jul 02 17:22:53.508415 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.508394 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.511782 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.511749 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c42a90b6-openshift-default-dockercfg-ddsdz\"" Jul 02 17:22:53.554971 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.554930 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw"] Jul 02 17:22:53.586018 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.585991 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-workload-socket\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.586018 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.586023 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istiod-ca-cert\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.586294 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.586052 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-credential-socket\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.586294 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.586075 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-token\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.586294 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.586215 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-data\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.586466 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.586294 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-workload-certs\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.586466 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.586331 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-podinfo\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.586466 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.586356 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6qwg\" (UniqueName: \"kubernetes.io/projected/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-kube-api-access-k6qwg\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.586466 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.586399 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-envoy\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.687693 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.687659 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-data\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.687892 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.687744 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-workload-certs\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.687892 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.687781 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-podinfo\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.687892 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.687812 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6qwg\" (UniqueName: \"kubernetes.io/projected/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-kube-api-access-k6qwg\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.687892 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.687843 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-envoy\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.687892 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.687884 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-workload-socket\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.688205 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.687911 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istiod-ca-cert\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.688205 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.687958 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-credential-socket\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.688205 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.688002 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-token\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.688205 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.688117 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-data\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.688205 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.688154 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-workload-certs\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.688475 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.688433 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-credential-socket\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.688541 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.688518 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-workload-socket\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.688693 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.688677 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istiod-ca-cert\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.690062 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.690044 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-envoy\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.690370 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.690347 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-podinfo\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.698999 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.698969 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6qwg\" (UniqueName: \"kubernetes.io/projected/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-kube-api-access-k6qwg\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.699196 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.699150 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-token\") pod \"e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.821900 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.821807 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:53.972435 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.972413 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw"] Jul 02 17:22:53.973753 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:22:53.973725 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod988b6ec0_b2cd_4174_97e0_03b6bd8f244c.slice/crio-50a53704bb2fa4b504dbf00d2006ad9e407052194093b0e3674e2acd35e4eb0a WatchSource:0}: Error finding container 50a53704bb2fa4b504dbf00d2006ad9e407052194093b0e3674e2acd35e4eb0a: Status 404 returned error can't find the container with id 50a53704bb2fa4b504dbf00d2006ad9e407052194093b0e3674e2acd35e4eb0a Jul 02 17:22:53.975824 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.975797 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:22:53.975896 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.975860 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:22:53.975896 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:53.975890 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:22:54.118504 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:54.118402 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" event={"ID":"988b6ec0-b2cd-4174-97e0-03b6bd8f244c","Type":"ContainerStarted","Data":"7c60411dd5ff79eb8e8c542908c8d6636f2a7343b2e7d8ec4179051f273cf29c"} Jul 02 17:22:54.118504 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:54.118455 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" event={"ID":"988b6ec0-b2cd-4174-97e0-03b6bd8f244c","Type":"ContainerStarted","Data":"50a53704bb2fa4b504dbf00d2006ad9e407052194093b0e3674e2acd35e4eb0a"} Jul 02 17:22:54.141933 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:54.141868 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" podStartSLOduration=1.141842547 podStartE2EDuration="1.141842547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:22:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:22:54.139873073 +0000 UTC m=+1707.452746722" watchObservedRunningTime="2026-07-02 17:22:54.141842547 +0000 UTC m=+1707.454716198" Jul 02 17:22:54.822549 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:54.822513 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:55.827408 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:55.827376 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:56.130306 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:56.130223 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:56.131364 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:56.131345 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:22:57.965186 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:22:57.965138 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:23:00.091629 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:00.091594 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw"] Jul 02 17:23:00.092023 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:00.091787 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" podUID="988b6ec0-b2cd-4174-97e0-03b6bd8f244c" containerName="istio-proxy" containerID="cri-o://7c60411dd5ff79eb8e8c542908c8d6636f2a7343b2e7d8ec4179051f273cf29c" gracePeriod=30 Jul 02 17:23:06.193191 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.193137 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr"] Jul 02 17:23:06.196554 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.196531 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.200875 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.200844 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e36c7a3e-openshift-default-dockercfg-zj97h\"" Jul 02 17:23:06.217656 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.217633 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr"] Jul 02 17:23:06.315451 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.315413 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-podinfo\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.315615 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.315492 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-envoy\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.315615 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.315544 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18a36f96-b5dd-450a-8d0d-5ed964c58972-istiod-ca-cert\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.315615 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.315581 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lfmt\" (UniqueName: \"kubernetes.io/projected/18a36f96-b5dd-450a-8d0d-5ed964c58972-kube-api-access-8lfmt\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.315783 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.315623 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-credential-socket\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.315783 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.315654 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-data\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.315783 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.315750 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-workload-socket\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.315931 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.315835 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-workload-certs\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.315931 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.315870 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-token\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.368226 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.368145 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:23:06.416638 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.416530 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-envoy\") pod \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " Jul 02 17:23:06.416638 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.416592 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-credential-socket\") pod \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " Jul 02 17:23:06.416638 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.416620 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-data\") pod \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " Jul 02 17:23:06.416638 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.416644 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-podinfo\") pod \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " Jul 02 17:23:06.416984 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.416681 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-workload-certs\") pod \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " Jul 02 17:23:06.416984 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.416744 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-workload-socket\") pod \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " Jul 02 17:23:06.416984 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.416767 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-token\") pod \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " Jul 02 17:23:06.416984 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.416834 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6qwg\" (UniqueName: \"kubernetes.io/projected/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-kube-api-access-k6qwg\") pod \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " Jul 02 17:23:06.416984 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.416871 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istiod-ca-cert\") pod \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\" (UID: \"988b6ec0-b2cd-4174-97e0-03b6bd8f244c\") " Jul 02 17:23:06.416984 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.416925 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "988b6ec0-b2cd-4174-97e0-03b6bd8f244c" (UID: "988b6ec0-b2cd-4174-97e0-03b6bd8f244c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:23:06.417326 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.417301 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "988b6ec0-b2cd-4174-97e0-03b6bd8f244c" (UID: "988b6ec0-b2cd-4174-97e0-03b6bd8f244c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:23:06.417537 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.417506 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "988b6ec0-b2cd-4174-97e0-03b6bd8f244c" (UID: "988b6ec0-b2cd-4174-97e0-03b6bd8f244c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:23:06.417632 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.417535 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-data" (OuterVolumeSpecName: "istio-data") pod "988b6ec0-b2cd-4174-97e0-03b6bd8f244c" (UID: "988b6ec0-b2cd-4174-97e0-03b6bd8f244c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:23:06.417632 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.417543 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "988b6ec0-b2cd-4174-97e0-03b6bd8f244c" (UID: "988b6ec0-b2cd-4174-97e0-03b6bd8f244c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:23:06.417632 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.417582 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-workload-certs\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.417632 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.417628 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-token\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.417832 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.417714 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-podinfo\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.417832 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.417771 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-envoy\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.417832 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.417807 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18a36f96-b5dd-450a-8d0d-5ed964c58972-istiod-ca-cert\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.417990 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.417847 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lfmt\" (UniqueName: \"kubernetes.io/projected/18a36f96-b5dd-450a-8d0d-5ed964c58972-kube-api-access-8lfmt\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.417990 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.417893 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-credential-socket\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.417990 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.417925 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-data\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.417990 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.417945 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-workload-certs\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.417990 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.417974 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-workload-socket\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.418293 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.418050 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istiod-ca-cert\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:06.418293 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.418067 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-credential-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:06.418293 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.418082 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-data\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:06.418293 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.418098 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-workload-certs\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:06.418293 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.418113 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-workload-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:06.418543 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.418383 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-workload-socket\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.418790 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.418764 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18a36f96-b5dd-450a-8d0d-5ed964c58972-istiod-ca-cert\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.418992 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.418962 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-credential-socket\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.419377 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.419333 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-data\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.420465 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.420075 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-token" (OuterVolumeSpecName: "istio-token") pod "988b6ec0-b2cd-4174-97e0-03b6bd8f244c" (UID: "988b6ec0-b2cd-4174-97e0-03b6bd8f244c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:23:06.421427 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.421395 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-kube-api-access-k6qwg" (OuterVolumeSpecName: "kube-api-access-k6qwg") pod "988b6ec0-b2cd-4174-97e0-03b6bd8f244c" (UID: "988b6ec0-b2cd-4174-97e0-03b6bd8f244c"). InnerVolumeSpecName "kube-api-access-k6qwg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:23:06.421652 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.421630 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-envoy\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.421787 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.421769 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "988b6ec0-b2cd-4174-97e0-03b6bd8f244c" (UID: "988b6ec0-b2cd-4174-97e0-03b6bd8f244c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:23:06.422382 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.422363 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-podinfo\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.422548 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.422525 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "988b6ec0-b2cd-4174-97e0-03b6bd8f244c" (UID: "988b6ec0-b2cd-4174-97e0-03b6bd8f244c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 17:23:06.426808 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.426781 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lfmt\" (UniqueName: \"kubernetes.io/projected/18a36f96-b5dd-450a-8d0d-5ed964c58972-kube-api-access-8lfmt\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.426887 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.426853 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-token\") pod \"e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.511947 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.511905 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:06.518926 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.518892 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6qwg\" (UniqueName: \"kubernetes.io/projected/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-kube-api-access-k6qwg\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:06.518926 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.518922 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-envoy\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:06.518926 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.518935 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-podinfo\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:06.519198 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.518943 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988b6ec0-b2cd-4174-97e0-03b6bd8f244c-istio-token\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:06.661910 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.661886 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr"] Jul 02 17:23:06.664120 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:23:06.664093 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18a36f96_b5dd_450a_8d0d_5ed964c58972.slice/crio-e8a8ae362f46effd72b423bce6b943f4e26392defbbe0c3e1982e71a0e6828fb WatchSource:0}: Error finding container e8a8ae362f46effd72b423bce6b943f4e26392defbbe0c3e1982e71a0e6828fb: Status 404 returned error can't find the container with id e8a8ae362f46effd72b423bce6b943f4e26392defbbe0c3e1982e71a0e6828fb Jul 02 17:23:06.666423 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.666393 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:23:06.666495 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.666455 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:23:06.666544 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:06.666492 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:23:07.197262 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:07.197227 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" event={"ID":"18a36f96-b5dd-450a-8d0d-5ed964c58972","Type":"ContainerStarted","Data":"a103adcd995e26c763908b2391849915b7221dd3e7d2ca88c9d9a364ecb55be4"} Jul 02 17:23:07.197262 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:07.197250 2556 generic.go:356] "Generic (PLEG): container finished" podID="988b6ec0-b2cd-4174-97e0-03b6bd8f244c" containerID="7c60411dd5ff79eb8e8c542908c8d6636f2a7343b2e7d8ec4179051f273cf29c" exitCode=0 Jul 02 17:23:07.197262 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:07.197271 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" event={"ID":"18a36f96-b5dd-450a-8d0d-5ed964c58972","Type":"ContainerStarted","Data":"e8a8ae362f46effd72b423bce6b943f4e26392defbbe0c3e1982e71a0e6828fb"} Jul 02 17:23:07.197874 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:07.197287 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" event={"ID":"988b6ec0-b2cd-4174-97e0-03b6bd8f244c","Type":"ContainerDied","Data":"7c60411dd5ff79eb8e8c542908c8d6636f2a7343b2e7d8ec4179051f273cf29c"} Jul 02 17:23:07.197874 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:07.197307 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" event={"ID":"988b6ec0-b2cd-4174-97e0-03b6bd8f244c","Type":"ContainerDied","Data":"50a53704bb2fa4b504dbf00d2006ad9e407052194093b0e3674e2acd35e4eb0a"} Jul 02 17:23:07.197874 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:07.197327 2556 scope.go:117] "RemoveContainer" containerID="7c60411dd5ff79eb8e8c542908c8d6636f2a7343b2e7d8ec4179051f273cf29c" Jul 02 17:23:07.197874 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:07.197338 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw" Jul 02 17:23:07.208184 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:07.207719 2556 scope.go:117] "RemoveContainer" containerID="7c60411dd5ff79eb8e8c542908c8d6636f2a7343b2e7d8ec4179051f273cf29c" Jul 02 17:23:07.208559 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:23:07.208530 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c60411dd5ff79eb8e8c542908c8d6636f2a7343b2e7d8ec4179051f273cf29c\": container with ID starting with 7c60411dd5ff79eb8e8c542908c8d6636f2a7343b2e7d8ec4179051f273cf29c not found: ID does not exist" containerID="7c60411dd5ff79eb8e8c542908c8d6636f2a7343b2e7d8ec4179051f273cf29c" Jul 02 17:23:07.208717 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:07.208567 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c60411dd5ff79eb8e8c542908c8d6636f2a7343b2e7d8ec4179051f273cf29c"} err="failed to get container status \"7c60411dd5ff79eb8e8c542908c8d6636f2a7343b2e7d8ec4179051f273cf29c\": rpc error: code = NotFound desc = could not find container \"7c60411dd5ff79eb8e8c542908c8d6636f2a7343b2e7d8ec4179051f273cf29c\": container with ID starting with 7c60411dd5ff79eb8e8c542908c8d6636f2a7343b2e7d8ec4179051f273cf29c not found: ID does not exist" Jul 02 17:23:07.217196 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:07.217146 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" podStartSLOduration=1.217134875 podStartE2EDuration="1.217134875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:23:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:23:07.214735987 +0000 UTC m=+1720.527609645" watchObservedRunningTime="2026-07-02 17:23:07.217134875 +0000 UTC m=+1720.530008521" Jul 02 17:23:07.229686 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:07.229663 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw"] Jul 02 17:23:07.233726 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:07.233706 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c42a90b6-openshift-default-85759f8565-phpfw"] Jul 02 17:23:07.512126 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:07.512098 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:07.517322 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:07.517295 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:08.204141 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:08.204107 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:08.205326 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:08.205302 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:09.191399 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:09.191365 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="988b6ec0-b2cd-4174-97e0-03b6bd8f244c" path="/var/lib/kubelet/pods/988b6ec0-b2cd-4174-97e0-03b6bd8f244c/volumes" Jul 02 17:23:10.563750 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:10.563713 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:23:18.264697 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:18.264665 2556 generic.go:356] "Generic (PLEG): container finished" podID="18a36f96-b5dd-450a-8d0d-5ed964c58972" containerID="a103adcd995e26c763908b2391849915b7221dd3e7d2ca88c9d9a364ecb55be4" exitCode=0 Jul 02 17:23:18.265122 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:18.264741 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" event={"ID":"18a36f96-b5dd-450a-8d0d-5ed964c58972","Type":"ContainerDied","Data":"a103adcd995e26c763908b2391849915b7221dd3e7d2ca88c9d9a364ecb55be4"} Jul 02 17:23:18.265122 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:18.265098 2556 scope.go:117] "RemoveContainer" containerID="a103adcd995e26c763908b2391849915b7221dd3e7d2ca88c9d9a364ecb55be4" Jul 02 17:23:18.266077 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:18.266051 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:23:18.266143 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:18.266118 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:23:18.266206 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:18.266154 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:23:19.015173 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:19.015131 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr"] Jul 02 17:23:19.271501 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:19.271407 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" event={"ID":"18a36f96-b5dd-450a-8d0d-5ed964c58972","Type":"ContainerStarted","Data":"ba479287e15a12b95df30f33e62d2aab681af7a134cdfb94fd352f721f48515b"} Jul 02 17:23:19.271501 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:19.271441 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" podUID="18a36f96-b5dd-450a-8d0d-5ed964c58972" containerName="istio-proxy" containerID="cri-o://ba479287e15a12b95df30f33e62d2aab681af7a134cdfb94fd352f721f48515b" gracePeriod=30 Jul 02 17:23:22.063833 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:22.063794 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:23:23.204976 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:23.204944 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:24.550208 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.550148 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:24.694285 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.694244 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-podinfo\") pod \"18a36f96-b5dd-450a-8d0d-5ed964c58972\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " Jul 02 17:23:24.694486 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.694310 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-data\") pod \"18a36f96-b5dd-450a-8d0d-5ed964c58972\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " Jul 02 17:23:24.694486 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.694341 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-workload-certs\") pod \"18a36f96-b5dd-450a-8d0d-5ed964c58972\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " Jul 02 17:23:24.694486 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.694363 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-workload-socket\") pod \"18a36f96-b5dd-450a-8d0d-5ed964c58972\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " Jul 02 17:23:24.694486 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.694379 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-token\") pod \"18a36f96-b5dd-450a-8d0d-5ed964c58972\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " Jul 02 17:23:24.694486 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.694406 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lfmt\" (UniqueName: \"kubernetes.io/projected/18a36f96-b5dd-450a-8d0d-5ed964c58972-kube-api-access-8lfmt\") pod \"18a36f96-b5dd-450a-8d0d-5ed964c58972\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " Jul 02 17:23:24.694486 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.694465 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-envoy\") pod \"18a36f96-b5dd-450a-8d0d-5ed964c58972\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " Jul 02 17:23:24.694807 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.694512 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18a36f96-b5dd-450a-8d0d-5ed964c58972-istiod-ca-cert\") pod \"18a36f96-b5dd-450a-8d0d-5ed964c58972\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " Jul 02 17:23:24.694807 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.694546 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-credential-socket\") pod \"18a36f96-b5dd-450a-8d0d-5ed964c58972\" (UID: \"18a36f96-b5dd-450a-8d0d-5ed964c58972\") " Jul 02 17:23:24.694807 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.694663 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "18a36f96-b5dd-450a-8d0d-5ed964c58972" (UID: "18a36f96-b5dd-450a-8d0d-5ed964c58972"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:23:24.694807 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.694681 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "18a36f96-b5dd-450a-8d0d-5ed964c58972" (UID: "18a36f96-b5dd-450a-8d0d-5ed964c58972"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:23:24.695016 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.694886 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "18a36f96-b5dd-450a-8d0d-5ed964c58972" (UID: "18a36f96-b5dd-450a-8d0d-5ed964c58972"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:23:24.695016 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.694943 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18a36f96-b5dd-450a-8d0d-5ed964c58972-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "18a36f96-b5dd-450a-8d0d-5ed964c58972" (UID: "18a36f96-b5dd-450a-8d0d-5ed964c58972"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:23:24.695016 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.694967 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-workload-certs\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:24.695016 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.694988 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-workload-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:24.695378 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.695355 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-data" (OuterVolumeSpecName: "istio-data") pod "18a36f96-b5dd-450a-8d0d-5ed964c58972" (UID: "18a36f96-b5dd-450a-8d0d-5ed964c58972"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:23:24.696839 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.696810 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "18a36f96-b5dd-450a-8d0d-5ed964c58972" (UID: "18a36f96-b5dd-450a-8d0d-5ed964c58972"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:23:24.696839 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.696825 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-token" (OuterVolumeSpecName: "istio-token") pod "18a36f96-b5dd-450a-8d0d-5ed964c58972" (UID: "18a36f96-b5dd-450a-8d0d-5ed964c58972"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:23:24.697024 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.697009 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "18a36f96-b5dd-450a-8d0d-5ed964c58972" (UID: "18a36f96-b5dd-450a-8d0d-5ed964c58972"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 17:23:24.697112 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.697097 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18a36f96-b5dd-450a-8d0d-5ed964c58972-kube-api-access-8lfmt" (OuterVolumeSpecName: "kube-api-access-8lfmt") pod "18a36f96-b5dd-450a-8d0d-5ed964c58972" (UID: "18a36f96-b5dd-450a-8d0d-5ed964c58972"). InnerVolumeSpecName "kube-api-access-8lfmt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:23:24.795559 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.795474 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-podinfo\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:24.795559 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.795504 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-data\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:24.795559 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.795515 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-token\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:24.795559 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.795524 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lfmt\" (UniqueName: \"kubernetes.io/projected/18a36f96-b5dd-450a-8d0d-5ed964c58972-kube-api-access-8lfmt\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:24.795559 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.795533 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-istio-envoy\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:24.795559 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.795542 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18a36f96-b5dd-450a-8d0d-5ed964c58972-istiod-ca-cert\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:24.795559 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:24.795550 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18a36f96-b5dd-450a-8d0d-5ed964c58972-credential-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:25.099563 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.099485 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6"] Jul 02 17:23:25.099966 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.099948 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="988b6ec0-b2cd-4174-97e0-03b6bd8f244c" containerName="istio-proxy" Jul 02 17:23:25.100050 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.099968 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="988b6ec0-b2cd-4174-97e0-03b6bd8f244c" containerName="istio-proxy" Jul 02 17:23:25.100050 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.099999 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18a36f96-b5dd-450a-8d0d-5ed964c58972" containerName="istio-proxy" Jul 02 17:23:25.100050 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.100008 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a36f96-b5dd-450a-8d0d-5ed964c58972" containerName="istio-proxy" Jul 02 17:23:25.100222 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.100132 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="988b6ec0-b2cd-4174-97e0-03b6bd8f244c" containerName="istio-proxy" Jul 02 17:23:25.100222 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.100146 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="18a36f96-b5dd-450a-8d0d-5ed964c58972" containerName="istio-proxy" Jul 02 17:23:25.100222 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.100199 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="18a36f96-b5dd-450a-8d0d-5ed964c58972" containerName="istio-proxy" Jul 02 17:23:25.102911 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.102887 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.105526 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.105506 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-a52f81ac-openshift-default-dockercfg-28czp\"" Jul 02 17:23:25.116690 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.116670 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6"] Jul 02 17:23:25.198044 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.198008 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-workload-certs\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.198238 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.198083 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-credential-socket\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.198238 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.198130 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-token\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.198238 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.198220 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/031993c7-6161-4dd0-9a09-9e5482754ed4-istiod-ca-cert\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.198780 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.198263 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-data\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.198780 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.198313 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-envoy\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.198780 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.198338 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7pxk\" (UniqueName: \"kubernetes.io/projected/031993c7-6161-4dd0-9a09-9e5482754ed4-kube-api-access-d7pxk\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.198780 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.198410 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-podinfo\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.198780 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.198444 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-workload-socket\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.299843 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.299795 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/031993c7-6161-4dd0-9a09-9e5482754ed4-istiod-ca-cert\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.300028 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.299848 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-data\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.300028 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.299890 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-envoy\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.300028 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.299912 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7pxk\" (UniqueName: \"kubernetes.io/projected/031993c7-6161-4dd0-9a09-9e5482754ed4-kube-api-access-d7pxk\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.300028 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.299973 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-podinfo\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.300028 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.300012 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-workload-socket\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.300271 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.300042 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-workload-certs\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.300271 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.300092 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-credential-socket\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.300271 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.300133 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-token\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.300366 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.300311 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-data\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.300794 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.300579 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-workload-certs\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.300794 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.300677 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/031993c7-6161-4dd0-9a09-9e5482754ed4-istiod-ca-cert\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.300794 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.300748 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-workload-socket\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.301020 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.300819 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-credential-socket\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.301707 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.301685 2556 generic.go:356] "Generic (PLEG): container finished" podID="18a36f96-b5dd-450a-8d0d-5ed964c58972" containerID="ba479287e15a12b95df30f33e62d2aab681af7a134cdfb94fd352f721f48515b" exitCode=0 Jul 02 17:23:25.301824 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.301753 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" Jul 02 17:23:25.301824 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.301778 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" event={"ID":"18a36f96-b5dd-450a-8d0d-5ed964c58972","Type":"ContainerDied","Data":"ba479287e15a12b95df30f33e62d2aab681af7a134cdfb94fd352f721f48515b"} Jul 02 17:23:25.301892 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.301854 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr" event={"ID":"18a36f96-b5dd-450a-8d0d-5ed964c58972","Type":"ContainerDied","Data":"e8a8ae362f46effd72b423bce6b943f4e26392defbbe0c3e1982e71a0e6828fb"} Jul 02 17:23:25.301929 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.301890 2556 scope.go:117] "RemoveContainer" containerID="ba479287e15a12b95df30f33e62d2aab681af7a134cdfb94fd352f721f48515b" Jul 02 17:23:25.302585 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.302565 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-envoy\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.302817 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.302797 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-podinfo\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.308188 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.308137 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7pxk\" (UniqueName: \"kubernetes.io/projected/031993c7-6161-4dd0-9a09-9e5482754ed4-kube-api-access-d7pxk\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.308276 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.308229 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-token\") pod \"e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.326816 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.326797 2556 scope.go:117] "RemoveContainer" containerID="a103adcd995e26c763908b2391849915b7221dd3e7d2ca88c9d9a364ecb55be4" Jul 02 17:23:25.335732 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.335712 2556 scope.go:117] "RemoveContainer" containerID="ba479287e15a12b95df30f33e62d2aab681af7a134cdfb94fd352f721f48515b" Jul 02 17:23:25.336016 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:23:25.335993 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba479287e15a12b95df30f33e62d2aab681af7a134cdfb94fd352f721f48515b\": container with ID starting with ba479287e15a12b95df30f33e62d2aab681af7a134cdfb94fd352f721f48515b not found: ID does not exist" containerID="ba479287e15a12b95df30f33e62d2aab681af7a134cdfb94fd352f721f48515b" Jul 02 17:23:25.336100 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.336023 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba479287e15a12b95df30f33e62d2aab681af7a134cdfb94fd352f721f48515b"} err="failed to get container status \"ba479287e15a12b95df30f33e62d2aab681af7a134cdfb94fd352f721f48515b\": rpc error: code = NotFound desc = could not find container \"ba479287e15a12b95df30f33e62d2aab681af7a134cdfb94fd352f721f48515b\": container with ID starting with ba479287e15a12b95df30f33e62d2aab681af7a134cdfb94fd352f721f48515b not found: ID does not exist" Jul 02 17:23:25.336100 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.336041 2556 scope.go:117] "RemoveContainer" containerID="a103adcd995e26c763908b2391849915b7221dd3e7d2ca88c9d9a364ecb55be4" Jul 02 17:23:25.336352 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:23:25.336328 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a103adcd995e26c763908b2391849915b7221dd3e7d2ca88c9d9a364ecb55be4\": container with ID starting with a103adcd995e26c763908b2391849915b7221dd3e7d2ca88c9d9a364ecb55be4 not found: ID does not exist" containerID="a103adcd995e26c763908b2391849915b7221dd3e7d2ca88c9d9a364ecb55be4" Jul 02 17:23:25.336416 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.336361 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a103adcd995e26c763908b2391849915b7221dd3e7d2ca88c9d9a364ecb55be4"} err="failed to get container status \"a103adcd995e26c763908b2391849915b7221dd3e7d2ca88c9d9a364ecb55be4\": rpc error: code = NotFound desc = could not find container \"a103adcd995e26c763908b2391849915b7221dd3e7d2ca88c9d9a364ecb55be4\": container with ID starting with a103adcd995e26c763908b2391849915b7221dd3e7d2ca88c9d9a364ecb55be4 not found: ID does not exist" Jul 02 17:23:25.338308 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.338290 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr"] Jul 02 17:23:25.340078 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.340060 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e36c7a3e-openshift-default-64cc54947c-gxhnr"] Jul 02 17:23:25.416583 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.416502 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:25.575832 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.575769 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6"] Jul 02 17:23:25.578333 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:23:25.578299 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod031993c7_6161_4dd0_9a09_9e5482754ed4.slice/crio-f361d1ffa4143a08b0106e7aae49ce72eb993629ca68f57661fe51155a974208 WatchSource:0}: Error finding container f361d1ffa4143a08b0106e7aae49ce72eb993629ca68f57661fe51155a974208: Status 404 returned error can't find the container with id f361d1ffa4143a08b0106e7aae49ce72eb993629ca68f57661fe51155a974208 Jul 02 17:23:25.580674 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.580635 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:23:25.580751 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.580729 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:23:25.580801 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:25.580782 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:23:26.313593 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:26.313563 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" event={"ID":"031993c7-6161-4dd0-9a09-9e5482754ed4","Type":"ContainerStarted","Data":"4c9bd5345cf717253977f6fe476c381aa0fe76dcff6fcc52692368e25b7e408a"} Jul 02 17:23:26.313751 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:26.313612 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" event={"ID":"031993c7-6161-4dd0-9a09-9e5482754ed4","Type":"ContainerStarted","Data":"f361d1ffa4143a08b0106e7aae49ce72eb993629ca68f57661fe51155a974208"} Jul 02 17:23:26.340434 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:26.340382 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" podStartSLOduration=1.340365163 podStartE2EDuration="1.340365163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:23:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:23:26.339896429 +0000 UTC m=+1739.652770076" watchObservedRunningTime="2026-07-02 17:23:26.340365163 +0000 UTC m=+1739.653238809" Jul 02 17:23:26.419865 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:26.418217 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:26.419865 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:26.418379 2556 patch_prober.go:28] interesting pod/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 02 17:23:26.419865 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:26.418424 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" podUID="031993c7-6161-4dd0-9a09-9e5482754ed4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 02 17:23:27.192018 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:27.191985 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18a36f96-b5dd-450a-8d0d-5ed964c58972" path="/var/lib/kubelet/pods/18a36f96-b5dd-450a-8d0d-5ed964c58972/volumes" Jul 02 17:23:27.422259 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:27.422231 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:28.324396 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:28.324357 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:28.325331 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:28.325301 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:29.955185 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:29.955131 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:23:30.994883 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:30.994850 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6"] Jul 02 17:23:31.340694 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:31.340630 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" podUID="031993c7-6161-4dd0-9a09-9e5482754ed4" containerName="istio-proxy" containerID="cri-o://4c9bd5345cf717253977f6fe476c381aa0fe76dcff6fcc52692368e25b7e408a" gracePeriod=30 Jul 02 17:23:37.699697 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.699669 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:37.713243 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.713192 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-credential-socket\") pod \"031993c7-6161-4dd0-9a09-9e5482754ed4\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " Jul 02 17:23:37.713243 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.713238 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-data\") pod \"031993c7-6161-4dd0-9a09-9e5482754ed4\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " Jul 02 17:23:37.713372 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.713268 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-token\") pod \"031993c7-6161-4dd0-9a09-9e5482754ed4\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " Jul 02 17:23:37.713372 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.713326 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/031993c7-6161-4dd0-9a09-9e5482754ed4-istiod-ca-cert\") pod \"031993c7-6161-4dd0-9a09-9e5482754ed4\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " Jul 02 17:23:37.713448 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.713377 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7pxk\" (UniqueName: \"kubernetes.io/projected/031993c7-6161-4dd0-9a09-9e5482754ed4-kube-api-access-d7pxk\") pod \"031993c7-6161-4dd0-9a09-9e5482754ed4\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " Jul 02 17:23:37.713448 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.713418 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-podinfo\") pod \"031993c7-6161-4dd0-9a09-9e5482754ed4\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " Jul 02 17:23:37.713531 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.713454 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-workload-certs\") pod \"031993c7-6161-4dd0-9a09-9e5482754ed4\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " Jul 02 17:23:37.713531 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.713492 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-envoy\") pod \"031993c7-6161-4dd0-9a09-9e5482754ed4\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " Jul 02 17:23:37.713531 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.713492 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "031993c7-6161-4dd0-9a09-9e5482754ed4" (UID: "031993c7-6161-4dd0-9a09-9e5482754ed4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:23:37.713531 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.713516 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-workload-socket\") pod \"031993c7-6161-4dd0-9a09-9e5482754ed4\" (UID: \"031993c7-6161-4dd0-9a09-9e5482754ed4\") " Jul 02 17:23:37.714062 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.713719 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "031993c7-6161-4dd0-9a09-9e5482754ed4" (UID: "031993c7-6161-4dd0-9a09-9e5482754ed4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:23:37.714062 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.713811 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-workload-certs\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:37.714062 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.713812 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/031993c7-6161-4dd0-9a09-9e5482754ed4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "031993c7-6161-4dd0-9a09-9e5482754ed4" (UID: "031993c7-6161-4dd0-9a09-9e5482754ed4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:23:37.714062 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.713829 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-credential-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:37.714062 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.713910 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "031993c7-6161-4dd0-9a09-9e5482754ed4" (UID: "031993c7-6161-4dd0-9a09-9e5482754ed4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:23:37.714430 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.714284 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-data" (OuterVolumeSpecName: "istio-data") pod "031993c7-6161-4dd0-9a09-9e5482754ed4" (UID: "031993c7-6161-4dd0-9a09-9e5482754ed4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:23:37.715799 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.715758 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "031993c7-6161-4dd0-9a09-9e5482754ed4" (UID: "031993c7-6161-4dd0-9a09-9e5482754ed4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 17:23:37.715905 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.715820 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "031993c7-6161-4dd0-9a09-9e5482754ed4" (UID: "031993c7-6161-4dd0-9a09-9e5482754ed4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:23:37.715905 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.715875 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/031993c7-6161-4dd0-9a09-9e5482754ed4-kube-api-access-d7pxk" (OuterVolumeSpecName: "kube-api-access-d7pxk") pod "031993c7-6161-4dd0-9a09-9e5482754ed4" (UID: "031993c7-6161-4dd0-9a09-9e5482754ed4"). InnerVolumeSpecName "kube-api-access-d7pxk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:23:37.715998 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.715906 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-token" (OuterVolumeSpecName: "istio-token") pod "031993c7-6161-4dd0-9a09-9e5482754ed4" (UID: "031993c7-6161-4dd0-9a09-9e5482754ed4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:23:37.815076 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.815037 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-envoy\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:37.815076 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.815071 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-workload-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:37.815076 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.815083 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-data\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:37.815360 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.815096 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-token\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:37.815360 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.815107 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/031993c7-6161-4dd0-9a09-9e5482754ed4-istiod-ca-cert\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:37.815360 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.815118 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7pxk\" (UniqueName: \"kubernetes.io/projected/031993c7-6161-4dd0-9a09-9e5482754ed4-kube-api-access-d7pxk\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:37.815360 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:37.815129 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/031993c7-6161-4dd0-9a09-9e5482754ed4-istio-podinfo\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:23:38.381784 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:38.381750 2556 generic.go:356] "Generic (PLEG): container finished" podID="031993c7-6161-4dd0-9a09-9e5482754ed4" containerID="4c9bd5345cf717253977f6fe476c381aa0fe76dcff6fcc52692368e25b7e408a" exitCode=0 Jul 02 17:23:38.381996 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:38.381807 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" Jul 02 17:23:38.381996 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:38.381817 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" event={"ID":"031993c7-6161-4dd0-9a09-9e5482754ed4","Type":"ContainerDied","Data":"4c9bd5345cf717253977f6fe476c381aa0fe76dcff6fcc52692368e25b7e408a"} Jul 02 17:23:38.381996 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:38.381862 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6" event={"ID":"031993c7-6161-4dd0-9a09-9e5482754ed4","Type":"ContainerDied","Data":"f361d1ffa4143a08b0106e7aae49ce72eb993629ca68f57661fe51155a974208"} Jul 02 17:23:38.381996 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:38.381883 2556 scope.go:117] "RemoveContainer" containerID="4c9bd5345cf717253977f6fe476c381aa0fe76dcff6fcc52692368e25b7e408a" Jul 02 17:23:38.391689 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:38.391672 2556 scope.go:117] "RemoveContainer" containerID="4c9bd5345cf717253977f6fe476c381aa0fe76dcff6fcc52692368e25b7e408a" Jul 02 17:23:38.391930 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:23:38.391911 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c9bd5345cf717253977f6fe476c381aa0fe76dcff6fcc52692368e25b7e408a\": container with ID starting with 4c9bd5345cf717253977f6fe476c381aa0fe76dcff6fcc52692368e25b7e408a not found: ID does not exist" containerID="4c9bd5345cf717253977f6fe476c381aa0fe76dcff6fcc52692368e25b7e408a" Jul 02 17:23:38.391976 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:38.391940 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c9bd5345cf717253977f6fe476c381aa0fe76dcff6fcc52692368e25b7e408a"} err="failed to get container status \"4c9bd5345cf717253977f6fe476c381aa0fe76dcff6fcc52692368e25b7e408a\": rpc error: code = NotFound desc = could not find container \"4c9bd5345cf717253977f6fe476c381aa0fe76dcff6fcc52692368e25b7e408a\": container with ID starting with 4c9bd5345cf717253977f6fe476c381aa0fe76dcff6fcc52692368e25b7e408a not found: ID does not exist" Jul 02 17:23:38.404713 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:38.404690 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6"] Jul 02 17:23:38.408783 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:38.408759 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-a52f81ac-openshift-default-7f5747dc8c-p92w6"] Jul 02 17:23:39.190270 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:39.190235 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="031993c7-6161-4dd0-9a09-9e5482754ed4" path="/var/lib/kubelet/pods/031993c7-6161-4dd0-9a09-9e5482754ed4/volumes" Jul 02 17:23:41.166336 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:23:41.166305 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:24:27.292951 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:24:27.292922 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 17:24:27.296352 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:24:27.296326 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 17:24:58.471811 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:24:58.471777 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:25:05.942608 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:05.942571 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:25:12.179139 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.179099 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm"] Jul 02 17:25:12.179585 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.179525 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18a36f96-b5dd-450a-8d0d-5ed964c58972" containerName="istio-proxy" Jul 02 17:25:12.179585 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.179537 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a36f96-b5dd-450a-8d0d-5ed964c58972" containerName="istio-proxy" Jul 02 17:25:12.179585 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.179546 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="031993c7-6161-4dd0-9a09-9e5482754ed4" containerName="istio-proxy" Jul 02 17:25:12.179585 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.179551 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="031993c7-6161-4dd0-9a09-9e5482754ed4" containerName="istio-proxy" Jul 02 17:25:12.179725 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.179613 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="031993c7-6161-4dd0-9a09-9e5482754ed4" containerName="istio-proxy" Jul 02 17:25:12.181810 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.181795 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.184538 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.184514 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7d46b667-openshift-default-dockercfg-pqvhm\"" Jul 02 17:25:12.208088 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.208061 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm"] Jul 02 17:25:12.361728 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.361695 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-podinfo\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.361904 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.361741 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-envoy\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.361904 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.361808 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd85d752-6b16-4424-a307-7167dd8c3e84-istiod-ca-cert\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.361904 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.361886 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-token\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.362009 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.361919 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-workload-socket\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.362009 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.361934 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-data\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.362009 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.361954 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh6hp\" (UniqueName: \"kubernetes.io/projected/dd85d752-6b16-4424-a307-7167dd8c3e84-kube-api-access-qh6hp\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.362130 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.362034 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-credential-socket\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.362130 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.362059 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-workload-certs\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.463319 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.463225 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-podinfo\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.463319 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.463273 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-envoy\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.463319 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.463297 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd85d752-6b16-4424-a307-7167dd8c3e84-istiod-ca-cert\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.463609 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.463327 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-token\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.463609 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.463351 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-workload-socket\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.463609 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.463365 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-data\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.463609 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.463384 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh6hp\" (UniqueName: \"kubernetes.io/projected/dd85d752-6b16-4424-a307-7167dd8c3e84-kube-api-access-qh6hp\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.463609 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.463475 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-credential-socket\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.463609 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.463526 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-workload-certs\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.463942 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.463919 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-data\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.464019 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.463964 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-credential-socket\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.464077 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.464042 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-workload-certs\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.464176 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.464140 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-workload-socket\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.464305 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.464281 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd85d752-6b16-4424-a307-7167dd8c3e84-istiod-ca-cert\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.465907 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.465884 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-envoy\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.465995 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.465961 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-podinfo\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.470944 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.470915 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-token\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.471067 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.471048 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh6hp\" (UniqueName: \"kubernetes.io/projected/dd85d752-6b16-4424-a307-7167dd8c3e84-kube-api-access-qh6hp\") pod \"e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.491952 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.491923 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:12.632079 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.632051 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm"] Jul 02 17:25:12.634366 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:25:12.634341 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd85d752_6b16_4424_a307_7167dd8c3e84.slice/crio-c2d91e25ad194b554fc56802f113faaab56d8d6a9dd0deaeaeeca17724590e68 WatchSource:0}: Error finding container c2d91e25ad194b554fc56802f113faaab56d8d6a9dd0deaeaeeca17724590e68: Status 404 returned error can't find the container with id c2d91e25ad194b554fc56802f113faaab56d8d6a9dd0deaeaeeca17724590e68 Jul 02 17:25:12.637086 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.637047 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:25:12.637190 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.637136 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:25:12.637245 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.637193 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:25:12.868980 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.868942 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" event={"ID":"dd85d752-6b16-4424-a307-7167dd8c3e84","Type":"ContainerStarted","Data":"bd7fd8563e7272146c6722c9f6dcdfe3c8d04629c0966778ab868ab15c455ef6"} Jul 02 17:25:12.869204 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.869010 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" event={"ID":"dd85d752-6b16-4424-a307-7167dd8c3e84","Type":"ContainerStarted","Data":"c2d91e25ad194b554fc56802f113faaab56d8d6a9dd0deaeaeeca17724590e68"} Jul 02 17:25:12.890264 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:12.890201 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" podStartSLOduration=0.890177084 podStartE2EDuration="890.177084ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:25:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:25:12.887654923 +0000 UTC m=+1846.200528565" watchObservedRunningTime="2026-07-02 17:25:12.890177084 +0000 UTC m=+1846.203050725" Jul 02 17:25:13.492065 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:13.492025 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:13.496871 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:13.496843 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:13.874567 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:13.874536 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:13.875639 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:13.875620 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:14.565490 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:14.565455 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:25:25.761278 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:25.761245 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:25:28.875746 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:28.875709 2556 patch_prober.go:28] interesting pod/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 02 17:25:28.876188 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:28.875779 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" podUID="dd85d752-6b16-4424-a307-7167dd8c3e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 02 17:25:28.961264 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:28.961231 2556 generic.go:356] "Generic (PLEG): container finished" podID="dd85d752-6b16-4424-a307-7167dd8c3e84" containerID="bd7fd8563e7272146c6722c9f6dcdfe3c8d04629c0966778ab868ab15c455ef6" exitCode=0 Jul 02 17:25:28.961437 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:28.961309 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" event={"ID":"dd85d752-6b16-4424-a307-7167dd8c3e84","Type":"ContainerDied","Data":"bd7fd8563e7272146c6722c9f6dcdfe3c8d04629c0966778ab868ab15c455ef6"} Jul 02 17:25:28.961727 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:28.961713 2556 scope.go:117] "RemoveContainer" containerID="bd7fd8563e7272146c6722c9f6dcdfe3c8d04629c0966778ab868ab15c455ef6" Jul 02 17:25:28.962797 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:28.962769 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:25:28.962871 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:28.962853 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:25:28.962921 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:28.962894 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:25:29.968322 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:29.968291 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" event={"ID":"dd85d752-6b16-4424-a307-7167dd8c3e84","Type":"ContainerStarted","Data":"e73e032deeaf268256d866cb912806f3a5c2744251fad39c0df43d08248cc708"} Jul 02 17:25:30.493024 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:30.492987 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:30.493264 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:30.493244 2556 patch_prober.go:28] interesting pod/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 02 17:25:30.493363 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:30.493298 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" podUID="dd85d752-6b16-4424-a307-7167dd8c3e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 02 17:25:31.492965 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:31.492929 2556 patch_prober.go:28] interesting pod/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 02 17:25:31.493394 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:31.492985 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" podUID="dd85d752-6b16-4424-a307-7167dd8c3e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 02 17:25:32.493227 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:32.493190 2556 patch_prober.go:28] interesting pod/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 02 17:25:32.493740 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:32.493244 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" podUID="dd85d752-6b16-4424-a307-7167dd8c3e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 02 17:25:33.249727 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:33.249693 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm"] Jul 02 17:25:33.250148 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:33.250120 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" podUID="dd85d752-6b16-4424-a307-7167dd8c3e84" containerName="istio-proxy" containerID="cri-o://e73e032deeaf268256d866cb912806f3a5c2744251fad39c0df43d08248cc708" gracePeriod=30 Jul 02 17:25:37.163207 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:37.163154 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:25:39.018339 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.018303 2556 generic.go:356] "Generic (PLEG): container finished" podID="dd85d752-6b16-4424-a307-7167dd8c3e84" containerID="e73e032deeaf268256d866cb912806f3a5c2744251fad39c0df43d08248cc708" exitCode=0 Jul 02 17:25:39.018716 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.018372 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" event={"ID":"dd85d752-6b16-4424-a307-7167dd8c3e84","Type":"ContainerDied","Data":"e73e032deeaf268256d866cb912806f3a5c2744251fad39c0df43d08248cc708"} Jul 02 17:25:39.018716 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.018458 2556 scope.go:117] "RemoveContainer" containerID="bd7fd8563e7272146c6722c9f6dcdfe3c8d04629c0966778ab868ab15c455ef6" Jul 02 17:25:39.115073 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.115043 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:39.218841 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.218753 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-envoy\") pod \"dd85d752-6b16-4424-a307-7167dd8c3e84\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " Jul 02 17:25:39.218841 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.218794 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-podinfo\") pod \"dd85d752-6b16-4424-a307-7167dd8c3e84\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " Jul 02 17:25:39.218841 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.218816 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-credential-socket\") pod \"dd85d752-6b16-4424-a307-7167dd8c3e84\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " Jul 02 17:25:39.218841 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.218841 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh6hp\" (UniqueName: \"kubernetes.io/projected/dd85d752-6b16-4424-a307-7167dd8c3e84-kube-api-access-qh6hp\") pod \"dd85d752-6b16-4424-a307-7167dd8c3e84\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " Jul 02 17:25:39.219290 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.218878 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-workload-socket\") pod \"dd85d752-6b16-4424-a307-7167dd8c3e84\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " Jul 02 17:25:39.219290 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.218934 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd85d752-6b16-4424-a307-7167dd8c3e84-istiod-ca-cert\") pod \"dd85d752-6b16-4424-a307-7167dd8c3e84\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " Jul 02 17:25:39.219290 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.219143 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dd85d752-6b16-4424-a307-7167dd8c3e84" (UID: "dd85d752-6b16-4424-a307-7167dd8c3e84"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:25:39.219290 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.219271 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-data\") pod \"dd85d752-6b16-4424-a307-7167dd8c3e84\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " Jul 02 17:25:39.219524 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.219371 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-token\") pod \"dd85d752-6b16-4424-a307-7167dd8c3e84\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " Jul 02 17:25:39.219524 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.219409 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-workload-certs\") pod \"dd85d752-6b16-4424-a307-7167dd8c3e84\" (UID: \"dd85d752-6b16-4424-a307-7167dd8c3e84\") " Jul 02 17:25:39.219524 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.219420 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd85d752-6b16-4424-a307-7167dd8c3e84-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dd85d752-6b16-4424-a307-7167dd8c3e84" (UID: "dd85d752-6b16-4424-a307-7167dd8c3e84"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:25:39.219731 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.219705 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dd85d752-6b16-4424-a307-7167dd8c3e84" (UID: "dd85d752-6b16-4424-a307-7167dd8c3e84"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:25:39.219785 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.219720 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-credential-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:25:39.219785 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.219756 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd85d752-6b16-4424-a307-7167dd8c3e84-istiod-ca-cert\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:25:39.219888 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.219810 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dd85d752-6b16-4424-a307-7167dd8c3e84" (UID: "dd85d752-6b16-4424-a307-7167dd8c3e84"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:25:39.220085 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.220064 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-data" (OuterVolumeSpecName: "istio-data") pod "dd85d752-6b16-4424-a307-7167dd8c3e84" (UID: "dd85d752-6b16-4424-a307-7167dd8c3e84"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:25:39.221248 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.221145 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dd85d752-6b16-4424-a307-7167dd8c3e84" (UID: "dd85d752-6b16-4424-a307-7167dd8c3e84"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:25:39.221399 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.221376 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dd85d752-6b16-4424-a307-7167dd8c3e84" (UID: "dd85d752-6b16-4424-a307-7167dd8c3e84"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 17:25:39.221732 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.221710 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-token" (OuterVolumeSpecName: "istio-token") pod "dd85d752-6b16-4424-a307-7167dd8c3e84" (UID: "dd85d752-6b16-4424-a307-7167dd8c3e84"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:25:39.221778 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.221730 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd85d752-6b16-4424-a307-7167dd8c3e84-kube-api-access-qh6hp" (OuterVolumeSpecName: "kube-api-access-qh6hp") pod "dd85d752-6b16-4424-a307-7167dd8c3e84" (UID: "dd85d752-6b16-4424-a307-7167dd8c3e84"). InnerVolumeSpecName "kube-api-access-qh6hp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:25:39.321483 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.321147 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-token\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:25:39.321483 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.321198 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-workload-certs\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:25:39.321483 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.321211 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-envoy\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:25:39.321483 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.321223 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-podinfo\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:25:39.321483 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.321236 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh6hp\" (UniqueName: \"kubernetes.io/projected/dd85d752-6b16-4424-a307-7167dd8c3e84-kube-api-access-qh6hp\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:25:39.321483 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.321250 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-workload-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:25:39.321483 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:39.321263 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd85d752-6b16-4424-a307-7167dd8c3e84-istio-data\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:25:40.025959 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:40.025909 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" event={"ID":"dd85d752-6b16-4424-a307-7167dd8c3e84","Type":"ContainerDied","Data":"c2d91e25ad194b554fc56802f113faaab56d8d6a9dd0deaeaeeca17724590e68"} Jul 02 17:25:40.025959 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:40.025927 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm" Jul 02 17:25:40.025959 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:40.025971 2556 scope.go:117] "RemoveContainer" containerID="e73e032deeaf268256d866cb912806f3a5c2744251fad39c0df43d08248cc708" Jul 02 17:25:40.047592 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:40.047564 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm"] Jul 02 17:25:40.049764 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:40.049744 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7d46b667-openshift-default-d7bb66bd7-z4crm"] Jul 02 17:25:41.190363 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:41.190323 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd85d752-6b16-4424-a307-7167dd8c3e84" path="/var/lib/kubelet/pods/dd85d752-6b16-4424-a307-7167dd8c3e84/volumes" Jul 02 17:25:51.930598 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:51.930515 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk"] Jul 02 17:25:51.931033 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:51.930911 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd85d752-6b16-4424-a307-7167dd8c3e84" containerName="istio-proxy" Jul 02 17:25:51.931033 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:51.930922 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd85d752-6b16-4424-a307-7167dd8c3e84" containerName="istio-proxy" Jul 02 17:25:51.931033 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:51.930996 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd85d752-6b16-4424-a307-7167dd8c3e84" containerName="istio-proxy" Jul 02 17:25:51.931033 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:51.931005 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd85d752-6b16-4424-a307-7167dd8c3e84" containerName="istio-proxy" Jul 02 17:25:51.934070 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:51.934040 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:51.937337 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:51.937314 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3245e5b9-openshift-default-dockercfg-nlqw8\"" Jul 02 17:25:51.949407 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:51.949329 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk"] Jul 02 17:25:52.036306 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.036259 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-workload-socket\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.036497 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.036312 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-workload-certs\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.036497 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.036358 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mpnt\" (UniqueName: \"kubernetes.io/projected/bd7db6eb-a028-4874-a56a-1d25c9463a63-kube-api-access-8mpnt\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.036497 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.036390 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd7db6eb-a028-4874-a56a-1d25c9463a63-istiod-ca-cert\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.036497 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.036418 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-envoy\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.036497 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.036444 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-podinfo\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.036497 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.036469 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-data\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.036826 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.036562 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-token\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.036826 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.036597 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-credential-socket\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.137639 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.137605 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-workload-socket\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.137639 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.137645 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-workload-certs\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.137882 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.137674 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mpnt\" (UniqueName: \"kubernetes.io/projected/bd7db6eb-a028-4874-a56a-1d25c9463a63-kube-api-access-8mpnt\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.137882 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.137697 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd7db6eb-a028-4874-a56a-1d25c9463a63-istiod-ca-cert\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.137882 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.137715 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-envoy\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.137882 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.137733 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-podinfo\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.137882 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.137753 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-data\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.137882 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.137783 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-token\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.137882 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.137811 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-credential-socket\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.138298 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.138095 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-workload-socket\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.138298 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.138143 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-credential-socket\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.138298 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.138220 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-data\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.138298 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.138272 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-workload-certs\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.138517 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.138459 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd7db6eb-a028-4874-a56a-1d25c9463a63-istiod-ca-cert\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.140127 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.140105 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-envoy\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.140256 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.140214 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-podinfo\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.146271 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.146253 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-token\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.146433 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.146415 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mpnt\" (UniqueName: \"kubernetes.io/projected/bd7db6eb-a028-4874-a56a-1d25c9463a63-kube-api-access-8mpnt\") pod \"e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.244711 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.244629 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:52.388452 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.388417 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk"] Jul 02 17:25:52.389567 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:25:52.389540 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd7db6eb_a028_4874_a56a_1d25c9463a63.slice/crio-631d7d2b1a80d42672d49ccd5f02a75a87e897f0faccf2458d420e4da25c8bf2 WatchSource:0}: Error finding container 631d7d2b1a80d42672d49ccd5f02a75a87e897f0faccf2458d420e4da25c8bf2: Status 404 returned error can't find the container with id 631d7d2b1a80d42672d49ccd5f02a75a87e897f0faccf2458d420e4da25c8bf2 Jul 02 17:25:52.391786 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.391758 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:25:52.391863 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.391822 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:25:52.391863 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:52.391849 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:25:53.097501 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:53.097459 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" event={"ID":"bd7db6eb-a028-4874-a56a-1d25c9463a63","Type":"ContainerStarted","Data":"49837937d87c0a0b4a54f0751d4800cc9988c63560c64a52cf18147769c6d91d"} Jul 02 17:25:53.097882 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:53.097514 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" event={"ID":"bd7db6eb-a028-4874-a56a-1d25c9463a63","Type":"ContainerStarted","Data":"631d7d2b1a80d42672d49ccd5f02a75a87e897f0faccf2458d420e4da25c8bf2"} Jul 02 17:25:53.120094 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:53.120045 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" podStartSLOduration=2.120029791 podStartE2EDuration="2.120029791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:25:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:25:53.119338308 +0000 UTC m=+1886.432211955" watchObservedRunningTime="2026-07-02 17:25:53.120029791 +0000 UTC m=+1886.432903437" Jul 02 17:25:53.245650 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:53.245617 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:53.250653 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:53.250627 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:54.103609 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:54.103574 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:25:54.104460 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:25:54.104445 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:26:01.065869 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:01.065835 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:26:09.105377 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:09.105271 2556 patch_prober.go:28] interesting pod/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 02 17:26:09.105377 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:09.105340 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" podUID="bd7db6eb-a028-4874-a56a-1d25c9463a63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 02 17:26:09.184435 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:09.184354 2556 generic.go:356] "Generic (PLEG): container finished" podID="bd7db6eb-a028-4874-a56a-1d25c9463a63" containerID="49837937d87c0a0b4a54f0751d4800cc9988c63560c64a52cf18147769c6d91d" exitCode=0 Jul 02 17:26:09.190350 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:09.190322 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" event={"ID":"bd7db6eb-a028-4874-a56a-1d25c9463a63","Type":"ContainerDied","Data":"49837937d87c0a0b4a54f0751d4800cc9988c63560c64a52cf18147769c6d91d"} Jul 02 17:26:09.190736 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:09.190720 2556 scope.go:117] "RemoveContainer" containerID="49837937d87c0a0b4a54f0751d4800cc9988c63560c64a52cf18147769c6d91d" Jul 02 17:26:09.191724 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:09.191692 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:26:09.191800 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:09.191766 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:26:09.191838 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:09.191805 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:26:10.191833 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:10.191802 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" event={"ID":"bd7db6eb-a028-4874-a56a-1d25c9463a63","Type":"ContainerStarted","Data":"b74502c973770fd3e2e9c63a1d945becf1d5f2428948621100885cc5b84ba37b"} Jul 02 17:26:10.245789 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:10.245747 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:26:10.246062 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:10.246041 2556 patch_prober.go:28] interesting pod/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 02 17:26:10.246129 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:10.246088 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" podUID="bd7db6eb-a028-4874-a56a-1d25c9463a63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 02 17:26:11.245486 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:11.245454 2556 patch_prober.go:28] interesting pod/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 02 17:26:11.245887 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:11.245507 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" podUID="bd7db6eb-a028-4874-a56a-1d25c9463a63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 02 17:26:11.383986 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:11.383954 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:26:12.245472 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:12.245438 2556 patch_prober.go:28] interesting pod/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 02 17:26:12.245680 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:12.245511 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" podUID="bd7db6eb-a028-4874-a56a-1d25c9463a63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 02 17:26:13.245748 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:13.245717 2556 patch_prober.go:28] interesting pod/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 02 17:26:13.246134 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:13.245777 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" podUID="bd7db6eb-a028-4874-a56a-1d25c9463a63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 02 17:26:14.245662 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:14.245623 2556 patch_prober.go:28] interesting pod/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 02 17:26:14.245859 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:14.245690 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" podUID="bd7db6eb-a028-4874-a56a-1d25c9463a63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 02 17:26:15.248747 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:15.248718 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:26:15.249191 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:15.249066 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:26:15.249840 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:15.249820 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:26:24.654470 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:24.654436 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:26:39.161573 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:26:39.161531 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:27:17.464498 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:17.464460 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:27:21.163637 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:21.163553 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk"] Jul 02 17:27:21.164084 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:21.163848 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" podUID="bd7db6eb-a028-4874-a56a-1d25c9463a63" containerName="istio-proxy" containerID="cri-o://b74502c973770fd3e2e9c63a1d945becf1d5f2428948621100885cc5b84ba37b" gracePeriod=30 Jul 02 17:27:26.424885 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.424816 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:27:26.496229 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.496196 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-podinfo\") pod \"bd7db6eb-a028-4874-a56a-1d25c9463a63\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " Jul 02 17:27:26.496390 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.496250 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-workload-socket\") pod \"bd7db6eb-a028-4874-a56a-1d25c9463a63\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " Jul 02 17:27:26.496390 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.496329 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd7db6eb-a028-4874-a56a-1d25c9463a63-istiod-ca-cert\") pod \"bd7db6eb-a028-4874-a56a-1d25c9463a63\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " Jul 02 17:27:26.496390 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.496357 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-data\") pod \"bd7db6eb-a028-4874-a56a-1d25c9463a63\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " Jul 02 17:27:26.496390 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.496387 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-workload-certs\") pod \"bd7db6eb-a028-4874-a56a-1d25c9463a63\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " Jul 02 17:27:26.496569 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.496405 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-envoy\") pod \"bd7db6eb-a028-4874-a56a-1d25c9463a63\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " Jul 02 17:27:26.496569 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.496455 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mpnt\" (UniqueName: \"kubernetes.io/projected/bd7db6eb-a028-4874-a56a-1d25c9463a63-kube-api-access-8mpnt\") pod \"bd7db6eb-a028-4874-a56a-1d25c9463a63\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " Jul 02 17:27:26.496569 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.496498 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-token\") pod \"bd7db6eb-a028-4874-a56a-1d25c9463a63\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " Jul 02 17:27:26.496569 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.496553 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-credential-socket\") pod \"bd7db6eb-a028-4874-a56a-1d25c9463a63\" (UID: \"bd7db6eb-a028-4874-a56a-1d25c9463a63\") " Jul 02 17:27:26.496785 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.496653 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bd7db6eb-a028-4874-a56a-1d25c9463a63" (UID: "bd7db6eb-a028-4874-a56a-1d25c9463a63"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:27:26.496785 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.496702 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd7db6eb-a028-4874-a56a-1d25c9463a63-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bd7db6eb-a028-4874-a56a-1d25c9463a63" (UID: "bd7db6eb-a028-4874-a56a-1d25c9463a63"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:27:26.496901 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.496846 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bd7db6eb-a028-4874-a56a-1d25c9463a63" (UID: "bd7db6eb-a028-4874-a56a-1d25c9463a63"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:27:26.496901 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.496885 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bd7db6eb-a028-4874-a56a-1d25c9463a63" (UID: "bd7db6eb-a028-4874-a56a-1d25c9463a63"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:27:26.497029 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.497011 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd7db6eb-a028-4874-a56a-1d25c9463a63-istiod-ca-cert\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:27:26.497096 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.497035 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-workload-certs\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:27:26.497096 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.497048 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-credential-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:27:26.497096 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.497056 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-workload-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:27:26.497288 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.497251 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-data" (OuterVolumeSpecName: "istio-data") pod "bd7db6eb-a028-4874-a56a-1d25c9463a63" (UID: "bd7db6eb-a028-4874-a56a-1d25c9463a63"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:27:26.498493 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.498462 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bd7db6eb-a028-4874-a56a-1d25c9463a63" (UID: "bd7db6eb-a028-4874-a56a-1d25c9463a63"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 17:27:26.498671 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.498648 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bd7db6eb-a028-4874-a56a-1d25c9463a63" (UID: "bd7db6eb-a028-4874-a56a-1d25c9463a63"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:27:26.498953 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.498930 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-token" (OuterVolumeSpecName: "istio-token") pod "bd7db6eb-a028-4874-a56a-1d25c9463a63" (UID: "bd7db6eb-a028-4874-a56a-1d25c9463a63"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:27:26.498953 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.498940 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd7db6eb-a028-4874-a56a-1d25c9463a63-kube-api-access-8mpnt" (OuterVolumeSpecName: "kube-api-access-8mpnt") pod "bd7db6eb-a028-4874-a56a-1d25c9463a63" (UID: "bd7db6eb-a028-4874-a56a-1d25c9463a63"). InnerVolumeSpecName "kube-api-access-8mpnt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:27:26.591471 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.591441 2556 generic.go:356] "Generic (PLEG): container finished" podID="bd7db6eb-a028-4874-a56a-1d25c9463a63" containerID="b74502c973770fd3e2e9c63a1d945becf1d5f2428948621100885cc5b84ba37b" exitCode=0 Jul 02 17:27:26.591629 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.591496 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" Jul 02 17:27:26.591629 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.591522 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" event={"ID":"bd7db6eb-a028-4874-a56a-1d25c9463a63","Type":"ContainerDied","Data":"b74502c973770fd3e2e9c63a1d945becf1d5f2428948621100885cc5b84ba37b"} Jul 02 17:27:26.591629 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.591576 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk" event={"ID":"bd7db6eb-a028-4874-a56a-1d25c9463a63","Type":"ContainerDied","Data":"631d7d2b1a80d42672d49ccd5f02a75a87e897f0faccf2458d420e4da25c8bf2"} Jul 02 17:27:26.591629 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.591598 2556 scope.go:117] "RemoveContainer" containerID="b74502c973770fd3e2e9c63a1d945becf1d5f2428948621100885cc5b84ba37b" Jul 02 17:27:26.597517 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.597497 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-envoy\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:27:26.597517 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.597518 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mpnt\" (UniqueName: \"kubernetes.io/projected/bd7db6eb-a028-4874-a56a-1d25c9463a63-kube-api-access-8mpnt\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:27:26.597657 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.597527 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-token\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:27:26.597657 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.597535 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-podinfo\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:27:26.597657 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.597543 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd7db6eb-a028-4874-a56a-1d25c9463a63-istio-data\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:27:26.600942 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.600927 2556 scope.go:117] "RemoveContainer" containerID="49837937d87c0a0b4a54f0751d4800cc9988c63560c64a52cf18147769c6d91d" Jul 02 17:27:26.611309 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.611292 2556 scope.go:117] "RemoveContainer" containerID="b74502c973770fd3e2e9c63a1d945becf1d5f2428948621100885cc5b84ba37b" Jul 02 17:27:26.611571 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:27:26.611550 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b74502c973770fd3e2e9c63a1d945becf1d5f2428948621100885cc5b84ba37b\": container with ID starting with b74502c973770fd3e2e9c63a1d945becf1d5f2428948621100885cc5b84ba37b not found: ID does not exist" containerID="b74502c973770fd3e2e9c63a1d945becf1d5f2428948621100885cc5b84ba37b" Jul 02 17:27:26.611617 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.611580 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b74502c973770fd3e2e9c63a1d945becf1d5f2428948621100885cc5b84ba37b"} err="failed to get container status \"b74502c973770fd3e2e9c63a1d945becf1d5f2428948621100885cc5b84ba37b\": rpc error: code = NotFound desc = could not find container \"b74502c973770fd3e2e9c63a1d945becf1d5f2428948621100885cc5b84ba37b\": container with ID starting with b74502c973770fd3e2e9c63a1d945becf1d5f2428948621100885cc5b84ba37b not found: ID does not exist" Jul 02 17:27:26.611617 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.611600 2556 scope.go:117] "RemoveContainer" containerID="49837937d87c0a0b4a54f0751d4800cc9988c63560c64a52cf18147769c6d91d" Jul 02 17:27:26.611843 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:27:26.611821 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49837937d87c0a0b4a54f0751d4800cc9988c63560c64a52cf18147769c6d91d\": container with ID starting with 49837937d87c0a0b4a54f0751d4800cc9988c63560c64a52cf18147769c6d91d not found: ID does not exist" containerID="49837937d87c0a0b4a54f0751d4800cc9988c63560c64a52cf18147769c6d91d" Jul 02 17:27:26.611883 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.611852 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49837937d87c0a0b4a54f0751d4800cc9988c63560c64a52cf18147769c6d91d"} err="failed to get container status \"49837937d87c0a0b4a54f0751d4800cc9988c63560c64a52cf18147769c6d91d\": rpc error: code = NotFound desc = could not find container \"49837937d87c0a0b4a54f0751d4800cc9988c63560c64a52cf18147769c6d91d\": container with ID starting with 49837937d87c0a0b4a54f0751d4800cc9988c63560c64a52cf18147769c6d91d not found: ID does not exist" Jul 02 17:27:26.616593 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.616571 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk"] Jul 02 17:27:26.622987 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:26.622967 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3245e5b9-openshift-default-99d487c6c-dsbpk"] Jul 02 17:27:27.190620 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:27.190589 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd7db6eb-a028-4874-a56a-1d25c9463a63" path="/var/lib/kubelet/pods/bd7db6eb-a028-4874-a56a-1d25c9463a63/volumes" Jul 02 17:27:31.770186 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:31.770141 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:27:41.157310 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:27:41.157275 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:28:15.861072 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.861042 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9"] Jul 02 17:28:15.861531 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.861487 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd7db6eb-a028-4874-a56a-1d25c9463a63" containerName="istio-proxy" Jul 02 17:28:15.861531 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.861503 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd7db6eb-a028-4874-a56a-1d25c9463a63" containerName="istio-proxy" Jul 02 17:28:15.861531 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.861520 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd7db6eb-a028-4874-a56a-1d25c9463a63" containerName="istio-proxy" Jul 02 17:28:15.861531 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.861525 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd7db6eb-a028-4874-a56a-1d25c9463a63" containerName="istio-proxy" Jul 02 17:28:15.861919 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.861535 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd85d752-6b16-4424-a307-7167dd8c3e84" containerName="istio-proxy" Jul 02 17:28:15.861919 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.861540 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd85d752-6b16-4424-a307-7167dd8c3e84" containerName="istio-proxy" Jul 02 17:28:15.861919 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.861616 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd7db6eb-a028-4874-a56a-1d25c9463a63" containerName="istio-proxy" Jul 02 17:28:15.861919 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.861628 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd7db6eb-a028-4874-a56a-1d25c9463a63" containerName="istio-proxy" Jul 02 17:28:15.863909 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.863889 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:15.867268 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.867249 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-bb1d81-openshift-default-dockercfg-kc8r4\"" Jul 02 17:28:15.878202 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.878177 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9"] Jul 02 17:28:15.935010 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.934978 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-workload-socket\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:15.935141 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.935014 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-istio-data\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:15.935141 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.935037 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-istio-envoy\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:15.935141 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.935118 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-workload-certs\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:15.935267 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.935199 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6116d61-38b8-429a-94d0-07b701ba1064-istio-podinfo\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:15.935267 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.935233 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nb7q\" (UniqueName: \"kubernetes.io/projected/b6116d61-38b8-429a-94d0-07b701ba1064-kube-api-access-2nb7q\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:15.935335 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.935265 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6116d61-38b8-429a-94d0-07b701ba1064-istio-token\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:15.935335 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.935285 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6116d61-38b8-429a-94d0-07b701ba1064-istiod-ca-cert\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:15.935401 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:15.935347 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-credential-socket\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.036571 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.036536 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6116d61-38b8-429a-94d0-07b701ba1064-istio-podinfo\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.036773 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.036596 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nb7q\" (UniqueName: \"kubernetes.io/projected/b6116d61-38b8-429a-94d0-07b701ba1064-kube-api-access-2nb7q\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.036773 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.036630 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6116d61-38b8-429a-94d0-07b701ba1064-istio-token\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.036773 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.036654 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6116d61-38b8-429a-94d0-07b701ba1064-istiod-ca-cert\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.036773 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.036710 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-credential-socket\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.036773 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.036773 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-workload-socket\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.037028 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.036804 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-istio-data\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.037028 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.036849 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-istio-envoy\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.037028 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.036888 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-workload-certs\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.037399 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.037375 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-istio-data\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.037504 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.037415 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-workload-socket\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.037504 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.037454 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-workload-certs\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.037792 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.037767 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-credential-socket\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.037869 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.037843 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6116d61-38b8-429a-94d0-07b701ba1064-istiod-ca-cert\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.039762 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.039741 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6116d61-38b8-429a-94d0-07b701ba1064-istio-podinfo\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.039877 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.039857 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-istio-envoy\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.046650 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.046623 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6116d61-38b8-429a-94d0-07b701ba1064-istio-token\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.046754 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.046740 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nb7q\" (UniqueName: \"kubernetes.io/projected/b6116d61-38b8-429a-94d0-07b701ba1064-kube-api-access-2nb7q\") pod \"e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.174792 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.174710 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:16.315271 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.315246 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9"] Jul 02 17:28:16.317583 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:28:16.317552 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6116d61_38b8_429a_94d0_07b701ba1064.slice/crio-ffa4efe8be00dbef9be3fc61fbd55a4aefc7f60eaa3dc1ee5af57d5618a03529 WatchSource:0}: Error finding container ffa4efe8be00dbef9be3fc61fbd55a4aefc7f60eaa3dc1ee5af57d5618a03529: Status 404 returned error can't find the container with id ffa4efe8be00dbef9be3fc61fbd55a4aefc7f60eaa3dc1ee5af57d5618a03529 Jul 02 17:28:16.319764 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.319729 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:28:16.319859 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.319803 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:28:16.319859 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.319833 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:28:16.860903 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.860868 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" event={"ID":"b6116d61-38b8-429a-94d0-07b701ba1064","Type":"ContainerStarted","Data":"078f1c38f26ce4713f356c1f0929b90296796266c0e48d955a1dc5e400b89845"} Jul 02 17:28:16.861078 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.860950 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" event={"ID":"b6116d61-38b8-429a-94d0-07b701ba1064","Type":"ContainerStarted","Data":"ffa4efe8be00dbef9be3fc61fbd55a4aefc7f60eaa3dc1ee5af57d5618a03529"} Jul 02 17:28:16.882847 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:16.882796 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" podStartSLOduration=1.882781086 podStartE2EDuration="1.882781086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:28:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:28:16.880416307 +0000 UTC m=+2030.193289953" watchObservedRunningTime="2026-07-02 17:28:16.882781086 +0000 UTC m=+2030.195654793" Jul 02 17:28:17.175191 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:17.175079 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:17.180176 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:17.180139 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:17.866503 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:17.866462 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:17.867421 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:17.867403 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:24.023905 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.023875 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd"] Jul 02 17:28:24.026649 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.026630 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.029355 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.029336 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-50b097-openshift-default-dockercfg-7qjts\"" Jul 02 17:28:24.038132 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.038109 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd"] Jul 02 17:28:24.111080 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.111044 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-data\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.111291 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.111092 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-credential-socket\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.111291 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.111200 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-workload-certs\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.111291 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.111242 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-envoy\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.111479 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.111298 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-workload-socket\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.111479 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.111342 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc527878-6689-4bee-aeeb-3aa8f556636a-istiod-ca-cert\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.111479 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.111407 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-token\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.111479 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.111444 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hxrw\" (UniqueName: \"kubernetes.io/projected/cc527878-6689-4bee-aeeb-3aa8f556636a-kube-api-access-5hxrw\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.111701 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.111486 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-podinfo\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.212925 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.212890 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-envoy\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.213121 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.212940 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-workload-socket\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.213121 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.213062 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc527878-6689-4bee-aeeb-3aa8f556636a-istiod-ca-cert\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.213304 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.213185 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-token\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.213304 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.213256 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hxrw\" (UniqueName: \"kubernetes.io/projected/cc527878-6689-4bee-aeeb-3aa8f556636a-kube-api-access-5hxrw\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.213418 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.213301 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-podinfo\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.213418 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.213320 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-workload-socket\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.213418 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.213373 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-data\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.213418 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.213415 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-credential-socket\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.213618 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.213470 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-workload-certs\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.213832 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.213773 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc527878-6689-4bee-aeeb-3aa8f556636a-istiod-ca-cert\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.213832 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.213797 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-workload-certs\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.214035 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.213968 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-credential-socket\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.214035 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.214018 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-data\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.215304 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.215285 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-envoy\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.216121 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.216103 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-podinfo\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.223308 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.223284 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-token\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.223420 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.223366 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hxrw\" (UniqueName: \"kubernetes.io/projected/cc527878-6689-4bee-aeeb-3aa8f556636a-kube-api-access-5hxrw\") pod \"e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.338038 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.337951 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:24.473207 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.473180 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd"] Jul 02 17:28:24.474276 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:28:24.474249 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc527878_6689_4bee_aeeb_3aa8f556636a.slice/crio-6856490c322842b453a7b2c22888abf32a0b5bbaac918d543c364e112d99e9b0 WatchSource:0}: Error finding container 6856490c322842b453a7b2c22888abf32a0b5bbaac918d543c364e112d99e9b0: Status 404 returned error can't find the container with id 6856490c322842b453a7b2c22888abf32a0b5bbaac918d543c364e112d99e9b0 Jul 02 17:28:24.476504 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.476475 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:28:24.476573 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.476540 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:28:24.476573 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.476570 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:28:24.904998 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.904955 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" event={"ID":"cc527878-6689-4bee-aeeb-3aa8f556636a","Type":"ContainerStarted","Data":"f9ef308d7eedcd712690f9f3defb2fa0e492c81466071762a4a70bceb9bccfa2"} Jul 02 17:28:24.905215 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.905026 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" event={"ID":"cc527878-6689-4bee-aeeb-3aa8f556636a","Type":"ContainerStarted","Data":"6856490c322842b453a7b2c22888abf32a0b5bbaac918d543c364e112d99e9b0"} Jul 02 17:28:24.927174 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:24.927107 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" podStartSLOduration=0.927091966 podStartE2EDuration="927.091966ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:28:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:28:24.925580693 +0000 UTC m=+2038.238454350" watchObservedRunningTime="2026-07-02 17:28:24.927091966 +0000 UTC m=+2038.239965612" Jul 02 17:28:25.338245 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:25.338204 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:25.343297 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:25.343272 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:25.911950 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:25.911916 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:25.913135 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:25.913114 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:26.613715 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:26.613682 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:28:33.908372 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:33.908328 2556 patch_prober.go:28] interesting pod/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 17:28:33.908841 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:33.908413 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" podUID="b6116d61-38b8-429a-94d0-07b701ba1064" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 17:28:34.974288 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:34.974251 2556 generic.go:356] "Generic (PLEG): container finished" podID="b6116d61-38b8-429a-94d0-07b701ba1064" containerID="078f1c38f26ce4713f356c1f0929b90296796266c0e48d955a1dc5e400b89845" exitCode=0 Jul 02 17:28:34.974765 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:34.974332 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" event={"ID":"b6116d61-38b8-429a-94d0-07b701ba1064","Type":"ContainerDied","Data":"078f1c38f26ce4713f356c1f0929b90296796266c0e48d955a1dc5e400b89845"} Jul 02 17:28:34.974765 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:34.974754 2556 scope.go:117] "RemoveContainer" containerID="078f1c38f26ce4713f356c1f0929b90296796266c0e48d955a1dc5e400b89845" Jul 02 17:28:34.975822 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:34.975797 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:28:34.975903 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:34.975865 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:28:34.975903 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:34.975897 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:28:35.981307 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:35.981268 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" event={"ID":"b6116d61-38b8-429a-94d0-07b701ba1064","Type":"ContainerStarted","Data":"15033f28fd273c07da3444d6130bbfdba1cf788aa50614587ec46bfca0f67e83"} Jul 02 17:28:36.175532 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:36.175498 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:36.175719 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:36.175605 2556 patch_prober.go:28] interesting pod/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 02 17:28:36.175719 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:36.175670 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" podUID="b6116d61-38b8-429a-94d0-07b701ba1064" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 02 17:28:37.175322 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:37.175279 2556 patch_prober.go:28] interesting pod/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 02 17:28:37.175814 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:37.175351 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" podUID="b6116d61-38b8-429a-94d0-07b701ba1064" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 02 17:28:38.175811 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:38.175772 2556 patch_prober.go:28] interesting pod/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 02 17:28:38.176237 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:38.175828 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" podUID="b6116d61-38b8-429a-94d0-07b701ba1064" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 02 17:28:39.176176 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:39.176123 2556 patch_prober.go:28] interesting pod/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 02 17:28:39.176571 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:39.176211 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" podUID="b6116d61-38b8-429a-94d0-07b701ba1064" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 02 17:28:40.178851 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:40.178818 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:40.179352 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:40.179332 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:40.180008 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:40.179988 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:28:40.760851 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:40.760809 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:28:41.954354 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:41.954313 2556 patch_prober.go:28] interesting pod/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 17:28:41.954770 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:41.954397 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" podUID="cc527878-6689-4bee-aeeb-3aa8f556636a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 17:28:43.021658 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:43.021625 2556 generic.go:356] "Generic (PLEG): container finished" podID="cc527878-6689-4bee-aeeb-3aa8f556636a" containerID="f9ef308d7eedcd712690f9f3defb2fa0e492c81466071762a4a70bceb9bccfa2" exitCode=0 Jul 02 17:28:43.022092 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:43.021703 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" event={"ID":"cc527878-6689-4bee-aeeb-3aa8f556636a","Type":"ContainerDied","Data":"f9ef308d7eedcd712690f9f3defb2fa0e492c81466071762a4a70bceb9bccfa2"} Jul 02 17:28:43.022235 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:43.022221 2556 scope.go:117] "RemoveContainer" containerID="f9ef308d7eedcd712690f9f3defb2fa0e492c81466071762a4a70bceb9bccfa2" Jul 02 17:28:43.023243 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:43.023218 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:28:43.023318 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:43.023284 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:28:43.023357 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:43.023317 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:28:44.034093 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:44.034042 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" event={"ID":"cc527878-6689-4bee-aeeb-3aa8f556636a","Type":"ContainerStarted","Data":"6d26715d501f46f6cbf8f03d98f965d02fd53b9517a1f9562be7cd8fd539e0bf"} Jul 02 17:28:44.339201 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:44.339075 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:44.339406 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:44.339381 2556 patch_prober.go:28] interesting pod/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 02 17:28:44.339456 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:44.339438 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" podUID="cc527878-6689-4bee-aeeb-3aa8f556636a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 02 17:28:45.338540 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:45.338497 2556 patch_prober.go:28] interesting pod/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 02 17:28:45.339059 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:45.338562 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" podUID="cc527878-6689-4bee-aeeb-3aa8f556636a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 02 17:28:46.338730 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:46.338687 2556 patch_prober.go:28] interesting pod/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 02 17:28:46.339255 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:46.338754 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" podUID="cc527878-6689-4bee-aeeb-3aa8f556636a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 02 17:28:47.338816 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:47.338778 2556 patch_prober.go:28] interesting pod/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 02 17:28:47.339249 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:47.338833 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" podUID="cc527878-6689-4bee-aeeb-3aa8f556636a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 02 17:28:48.342029 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:48.341999 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:48.342411 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:48.342364 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:48.342899 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:48.342882 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:28:56.562639 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:28:56.562592 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:29:07.659449 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:29:07.659410 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:29:27.327790 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:29:27.327761 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 17:29:27.332308 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:29:27.332287 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 17:29:49.062850 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:29:49.062812 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:30:04.862073 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:30:04.862039 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:30:16.566829 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:30:16.566795 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:30:54.789888 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:30:54.789847 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9"] Jul 02 17:30:54.790584 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:30:54.790117 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" podUID="b6116d61-38b8-429a-94d0-07b701ba1064" containerName="istio-proxy" containerID="cri-o://15033f28fd273c07da3444d6130bbfdba1cf788aa50614587ec46bfca0f67e83" gracePeriod=30 Jul 02 17:30:55.182444 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:30:55.182352 2556 patch_prober.go:28] interesting pod/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 17:30:55.182444 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:30:55.182417 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" podUID="b6116d61-38b8-429a-94d0-07b701ba1064" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 17:30:58.859328 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:30:58.859295 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:31:00.062371 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.062312 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:31:00.167950 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.167915 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6116d61-38b8-429a-94d0-07b701ba1064-istio-podinfo\") pod \"b6116d61-38b8-429a-94d0-07b701ba1064\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " Jul 02 17:31:00.168117 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.167976 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nb7q\" (UniqueName: \"kubernetes.io/projected/b6116d61-38b8-429a-94d0-07b701ba1064-kube-api-access-2nb7q\") pod \"b6116d61-38b8-429a-94d0-07b701ba1064\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " Jul 02 17:31:00.168117 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.168006 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-istio-data\") pod \"b6116d61-38b8-429a-94d0-07b701ba1064\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " Jul 02 17:31:00.168117 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.168061 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-workload-socket\") pod \"b6116d61-38b8-429a-94d0-07b701ba1064\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " Jul 02 17:31:00.168270 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.168186 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-credential-socket\") pod \"b6116d61-38b8-429a-94d0-07b701ba1064\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " Jul 02 17:31:00.168270 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.168245 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-istio-envoy\") pod \"b6116d61-38b8-429a-94d0-07b701ba1064\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " Jul 02 17:31:00.168367 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.168274 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6116d61-38b8-429a-94d0-07b701ba1064-istio-token\") pod \"b6116d61-38b8-429a-94d0-07b701ba1064\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " Jul 02 17:31:00.168367 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.168329 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-workload-certs\") pod \"b6116d61-38b8-429a-94d0-07b701ba1064\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " Jul 02 17:31:00.168367 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.168333 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b6116d61-38b8-429a-94d0-07b701ba1064" (UID: "b6116d61-38b8-429a-94d0-07b701ba1064"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:00.168367 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.168363 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6116d61-38b8-429a-94d0-07b701ba1064-istiod-ca-cert\") pod \"b6116d61-38b8-429a-94d0-07b701ba1064\" (UID: \"b6116d61-38b8-429a-94d0-07b701ba1064\") " Jul 02 17:31:00.168575 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.168413 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b6116d61-38b8-429a-94d0-07b701ba1064" (UID: "b6116d61-38b8-429a-94d0-07b701ba1064"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:00.168575 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.168501 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b6116d61-38b8-429a-94d0-07b701ba1064" (UID: "b6116d61-38b8-429a-94d0-07b701ba1064"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:00.168712 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.168693 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-workload-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:00.168773 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.168718 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-credential-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:00.168773 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.168732 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-workload-certs\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:00.168875 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.168796 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6116d61-38b8-429a-94d0-07b701ba1064-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b6116d61-38b8-429a-94d0-07b701ba1064" (UID: "b6116d61-38b8-429a-94d0-07b701ba1064"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:31:00.169125 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.169097 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-istio-data" (OuterVolumeSpecName: "istio-data") pod "b6116d61-38b8-429a-94d0-07b701ba1064" (UID: "b6116d61-38b8-429a-94d0-07b701ba1064"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:00.170320 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.170293 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6116d61-38b8-429a-94d0-07b701ba1064-kube-api-access-2nb7q" (OuterVolumeSpecName: "kube-api-access-2nb7q") pod "b6116d61-38b8-429a-94d0-07b701ba1064" (UID: "b6116d61-38b8-429a-94d0-07b701ba1064"). InnerVolumeSpecName "kube-api-access-2nb7q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:31:00.170456 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.170432 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6116d61-38b8-429a-94d0-07b701ba1064-istio-token" (OuterVolumeSpecName: "istio-token") pod "b6116d61-38b8-429a-94d0-07b701ba1064" (UID: "b6116d61-38b8-429a-94d0-07b701ba1064"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:31:00.170530 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.170495 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b6116d61-38b8-429a-94d0-07b701ba1064" (UID: "b6116d61-38b8-429a-94d0-07b701ba1064"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:00.170580 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.170506 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6116d61-38b8-429a-94d0-07b701ba1064-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b6116d61-38b8-429a-94d0-07b701ba1064" (UID: "b6116d61-38b8-429a-94d0-07b701ba1064"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 17:31:00.269191 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.269140 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nb7q\" (UniqueName: \"kubernetes.io/projected/b6116d61-38b8-429a-94d0-07b701ba1064-kube-api-access-2nb7q\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:00.269191 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.269188 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-istio-data\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:00.269191 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.269198 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6116d61-38b8-429a-94d0-07b701ba1064-istio-envoy\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:00.269410 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.269206 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6116d61-38b8-429a-94d0-07b701ba1064-istio-token\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:00.269410 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.269214 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6116d61-38b8-429a-94d0-07b701ba1064-istiod-ca-cert\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:00.269410 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.269221 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6116d61-38b8-429a-94d0-07b701ba1064-istio-podinfo\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:00.752484 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.752446 2556 generic.go:356] "Generic (PLEG): container finished" podID="b6116d61-38b8-429a-94d0-07b701ba1064" containerID="15033f28fd273c07da3444d6130bbfdba1cf788aa50614587ec46bfca0f67e83" exitCode=0 Jul 02 17:31:00.752667 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.752498 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" Jul 02 17:31:00.752667 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.752499 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" event={"ID":"b6116d61-38b8-429a-94d0-07b701ba1064","Type":"ContainerDied","Data":"15033f28fd273c07da3444d6130bbfdba1cf788aa50614587ec46bfca0f67e83"} Jul 02 17:31:00.752667 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.752620 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9" event={"ID":"b6116d61-38b8-429a-94d0-07b701ba1064","Type":"ContainerDied","Data":"ffa4efe8be00dbef9be3fc61fbd55a4aefc7f60eaa3dc1ee5af57d5618a03529"} Jul 02 17:31:00.752667 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.752650 2556 scope.go:117] "RemoveContainer" containerID="15033f28fd273c07da3444d6130bbfdba1cf788aa50614587ec46bfca0f67e83" Jul 02 17:31:00.764952 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.764936 2556 scope.go:117] "RemoveContainer" containerID="078f1c38f26ce4713f356c1f0929b90296796266c0e48d955a1dc5e400b89845" Jul 02 17:31:00.773397 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.773374 2556 scope.go:117] "RemoveContainer" containerID="15033f28fd273c07da3444d6130bbfdba1cf788aa50614587ec46bfca0f67e83" Jul 02 17:31:00.773759 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:31:00.773740 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15033f28fd273c07da3444d6130bbfdba1cf788aa50614587ec46bfca0f67e83\": container with ID starting with 15033f28fd273c07da3444d6130bbfdba1cf788aa50614587ec46bfca0f67e83 not found: ID does not exist" containerID="15033f28fd273c07da3444d6130bbfdba1cf788aa50614587ec46bfca0f67e83" Jul 02 17:31:00.773840 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.773767 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15033f28fd273c07da3444d6130bbfdba1cf788aa50614587ec46bfca0f67e83"} err="failed to get container status \"15033f28fd273c07da3444d6130bbfdba1cf788aa50614587ec46bfca0f67e83\": rpc error: code = NotFound desc = could not find container \"15033f28fd273c07da3444d6130bbfdba1cf788aa50614587ec46bfca0f67e83\": container with ID starting with 15033f28fd273c07da3444d6130bbfdba1cf788aa50614587ec46bfca0f67e83 not found: ID does not exist" Jul 02 17:31:00.773840 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.773784 2556 scope.go:117] "RemoveContainer" containerID="078f1c38f26ce4713f356c1f0929b90296796266c0e48d955a1dc5e400b89845" Jul 02 17:31:00.774075 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:31:00.774056 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"078f1c38f26ce4713f356c1f0929b90296796266c0e48d955a1dc5e400b89845\": container with ID starting with 078f1c38f26ce4713f356c1f0929b90296796266c0e48d955a1dc5e400b89845 not found: ID does not exist" containerID="078f1c38f26ce4713f356c1f0929b90296796266c0e48d955a1dc5e400b89845" Jul 02 17:31:00.774128 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.774079 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"078f1c38f26ce4713f356c1f0929b90296796266c0e48d955a1dc5e400b89845"} err="failed to get container status \"078f1c38f26ce4713f356c1f0929b90296796266c0e48d955a1dc5e400b89845\": rpc error: code = NotFound desc = could not find container \"078f1c38f26ce4713f356c1f0929b90296796266c0e48d955a1dc5e400b89845\": container with ID starting with 078f1c38f26ce4713f356c1f0929b90296796266c0e48d955a1dc5e400b89845 not found: ID does not exist" Jul 02 17:31:00.776042 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.776023 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9"] Jul 02 17:31:00.779568 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:00.779546 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-bb1d81-openshift-default-58d944fc9c-crvm9"] Jul 02 17:31:01.190756 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:01.190667 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6116d61-38b8-429a-94d0-07b701ba1064" path="/var/lib/kubelet/pods/b6116d61-38b8-429a-94d0-07b701ba1064/volumes" Jul 02 17:31:01.807101 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:01.807065 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd"] Jul 02 17:31:01.807425 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:01.807398 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" podUID="cc527878-6689-4bee-aeeb-3aa8f556636a" containerName="istio-proxy" containerID="cri-o://6d26715d501f46f6cbf8f03d98f965d02fd53b9517a1f9562be7cd8fd539e0bf" gracePeriod=30 Jul 02 17:31:02.855751 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.855720 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw"] Jul 02 17:31:02.856173 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.856146 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6116d61-38b8-429a-94d0-07b701ba1064" containerName="istio-proxy" Jul 02 17:31:02.856245 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.856182 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6116d61-38b8-429a-94d0-07b701ba1064" containerName="istio-proxy" Jul 02 17:31:02.856245 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.856193 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6116d61-38b8-429a-94d0-07b701ba1064" containerName="istio-proxy" Jul 02 17:31:02.856245 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.856198 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6116d61-38b8-429a-94d0-07b701ba1064" containerName="istio-proxy" Jul 02 17:31:02.856361 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.856263 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6116d61-38b8-429a-94d0-07b701ba1064" containerName="istio-proxy" Jul 02 17:31:02.859591 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.859569 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:02.862231 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.862207 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-28df41-openshift-default-dockercfg-p8w5d\"" Jul 02 17:31:02.872332 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.872309 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw"] Jul 02 17:31:02.990855 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.990821 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-envoy\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:02.990855 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.990859 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34637434-e8b2-431a-b224-c2d5bd4816ff-istiod-ca-cert\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:02.991074 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.990896 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck6qj\" (UniqueName: \"kubernetes.io/projected/34637434-e8b2-431a-b224-c2d5bd4816ff-kube-api-access-ck6qj\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:02.991074 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.990921 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-credential-socket\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:02.991074 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.990937 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-data\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:02.991074 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.990966 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-workload-certs\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:02.991262 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.991064 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-podinfo\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:02.991262 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.991118 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-workload-socket\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:02.991262 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:02.991140 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-token\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.092473 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.092435 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-workload-certs\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.092705 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.092499 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-podinfo\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.092705 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.092536 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-workload-socket\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.092705 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.092554 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-token\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.092705 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.092606 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-envoy\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.092705 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.092632 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34637434-e8b2-431a-b224-c2d5bd4816ff-istiod-ca-cert\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.092705 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.092666 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck6qj\" (UniqueName: \"kubernetes.io/projected/34637434-e8b2-431a-b224-c2d5bd4816ff-kube-api-access-ck6qj\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.092705 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.092696 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-credential-socket\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.093078 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.092719 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-data\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.093078 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.092965 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-workload-certs\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.093192 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.093129 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-credential-socket\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.093424 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.093404 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-workload-socket\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.093482 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.093464 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34637434-e8b2-431a-b224-c2d5bd4816ff-istiod-ca-cert\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.093482 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.093472 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-data\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.094974 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.094943 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-envoy\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.095078 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.095016 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-podinfo\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.100506 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.100483 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-token\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.100633 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.100615 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck6qj\" (UniqueName: \"kubernetes.io/projected/34637434-e8b2-431a-b224-c2d5bd4816ff-kube-api-access-ck6qj\") pod \"e2e-api-a-28df41-openshift-default-6486665885-wrxpw\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.170790 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.170687 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:03.319824 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.319795 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw"] Jul 02 17:31:03.321522 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:31:03.321489 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34637434_e8b2_431a_b224_c2d5bd4816ff.slice/crio-231229c6f324085b35fe7084c1f3cd7f3bf25b34d3da1938bdce477a79c31b6e WatchSource:0}: Error finding container 231229c6f324085b35fe7084c1f3cd7f3bf25b34d3da1938bdce477a79c31b6e: Status 404 returned error can't find the container with id 231229c6f324085b35fe7084c1f3cd7f3bf25b34d3da1938bdce477a79c31b6e Jul 02 17:31:03.323641 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.323609 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:03.323741 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.323671 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:03.323741 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.323698 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:03.345077 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.345055 2556 patch_prober.go:28] interesting pod/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 17:31:03.345189 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.345094 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" podUID="cc527878-6689-4bee-aeeb-3aa8f556636a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 17:31:03.771993 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.771956 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" event={"ID":"34637434-e8b2-431a-b224-c2d5bd4816ff","Type":"ContainerStarted","Data":"e37a830fbbe5fe9eefa038a29b81bea447d76c049de9a42352819ccc94b2dd6b"} Jul 02 17:31:03.772191 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.772014 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" event={"ID":"34637434-e8b2-431a-b224-c2d5bd4816ff","Type":"ContainerStarted","Data":"231229c6f324085b35fe7084c1f3cd7f3bf25b34d3da1938bdce477a79c31b6e"} Jul 02 17:31:03.793036 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:03.792991 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" podStartSLOduration=1.792974449 podStartE2EDuration="1.792974449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:31:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:31:03.791983387 +0000 UTC m=+2197.104857035" watchObservedRunningTime="2026-07-02 17:31:03.792974449 +0000 UTC m=+2197.105848095" Jul 02 17:31:04.171603 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:04.171521 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:04.176372 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:04.176348 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:04.776938 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:04.776905 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:04.777874 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:04.777852 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:07.071298 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.071223 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:31:07.125421 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.125391 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-envoy\") pod \"cc527878-6689-4bee-aeeb-3aa8f556636a\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " Jul 02 17:31:07.125587 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.125448 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-data\") pod \"cc527878-6689-4bee-aeeb-3aa8f556636a\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " Jul 02 17:31:07.125587 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.125486 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-podinfo\") pod \"cc527878-6689-4bee-aeeb-3aa8f556636a\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " Jul 02 17:31:07.125587 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.125511 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-workload-certs\") pod \"cc527878-6689-4bee-aeeb-3aa8f556636a\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " Jul 02 17:31:07.125587 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.125540 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-credential-socket\") pod \"cc527878-6689-4bee-aeeb-3aa8f556636a\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " Jul 02 17:31:07.125587 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.125567 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc527878-6689-4bee-aeeb-3aa8f556636a-istiod-ca-cert\") pod \"cc527878-6689-4bee-aeeb-3aa8f556636a\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " Jul 02 17:31:07.125858 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.125592 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-token\") pod \"cc527878-6689-4bee-aeeb-3aa8f556636a\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " Jul 02 17:31:07.125858 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.125649 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-workload-socket\") pod \"cc527878-6689-4bee-aeeb-3aa8f556636a\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " Jul 02 17:31:07.125858 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.125726 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hxrw\" (UniqueName: \"kubernetes.io/projected/cc527878-6689-4bee-aeeb-3aa8f556636a-kube-api-access-5hxrw\") pod \"cc527878-6689-4bee-aeeb-3aa8f556636a\" (UID: \"cc527878-6689-4bee-aeeb-3aa8f556636a\") " Jul 02 17:31:07.126421 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.125857 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cc527878-6689-4bee-aeeb-3aa8f556636a" (UID: "cc527878-6689-4bee-aeeb-3aa8f556636a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:07.126421 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.125872 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cc527878-6689-4bee-aeeb-3aa8f556636a" (UID: "cc527878-6689-4bee-aeeb-3aa8f556636a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:07.126421 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.126065 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cc527878-6689-4bee-aeeb-3aa8f556636a" (UID: "cc527878-6689-4bee-aeeb-3aa8f556636a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:07.126421 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.126219 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc527878-6689-4bee-aeeb-3aa8f556636a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cc527878-6689-4bee-aeeb-3aa8f556636a" (UID: "cc527878-6689-4bee-aeeb-3aa8f556636a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:31:07.126421 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.126301 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-workload-certs\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:07.126421 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.126323 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-credential-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:07.126421 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.126359 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc527878-6689-4bee-aeeb-3aa8f556636a-istiod-ca-cert\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:07.126421 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.126375 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-workload-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:07.126867 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.126471 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-data" (OuterVolumeSpecName: "istio-data") pod "cc527878-6689-4bee-aeeb-3aa8f556636a" (UID: "cc527878-6689-4bee-aeeb-3aa8f556636a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:07.127917 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.127891 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cc527878-6689-4bee-aeeb-3aa8f556636a" (UID: "cc527878-6689-4bee-aeeb-3aa8f556636a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 17:31:07.128314 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.128294 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cc527878-6689-4bee-aeeb-3aa8f556636a" (UID: "cc527878-6689-4bee-aeeb-3aa8f556636a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:07.128508 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.128489 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc527878-6689-4bee-aeeb-3aa8f556636a-kube-api-access-5hxrw" (OuterVolumeSpecName: "kube-api-access-5hxrw") pod "cc527878-6689-4bee-aeeb-3aa8f556636a" (UID: "cc527878-6689-4bee-aeeb-3aa8f556636a"). InnerVolumeSpecName "kube-api-access-5hxrw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:31:07.128562 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.128539 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-token" (OuterVolumeSpecName: "istio-token") pod "cc527878-6689-4bee-aeeb-3aa8f556636a" (UID: "cc527878-6689-4bee-aeeb-3aa8f556636a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:31:07.227269 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.227238 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-envoy\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:07.227269 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.227264 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-data\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:07.227269 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.227276 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-podinfo\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:07.227527 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.227286 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc527878-6689-4bee-aeeb-3aa8f556636a-istio-token\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:07.227527 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.227318 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hxrw\" (UniqueName: \"kubernetes.io/projected/cc527878-6689-4bee-aeeb-3aa8f556636a-kube-api-access-5hxrw\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:07.801176 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.801123 2556 generic.go:356] "Generic (PLEG): container finished" podID="cc527878-6689-4bee-aeeb-3aa8f556636a" containerID="6d26715d501f46f6cbf8f03d98f965d02fd53b9517a1f9562be7cd8fd539e0bf" exitCode=0 Jul 02 17:31:07.801346 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.801185 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" event={"ID":"cc527878-6689-4bee-aeeb-3aa8f556636a","Type":"ContainerDied","Data":"6d26715d501f46f6cbf8f03d98f965d02fd53b9517a1f9562be7cd8fd539e0bf"} Jul 02 17:31:07.801346 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.801192 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" Jul 02 17:31:07.801346 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.801233 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd" event={"ID":"cc527878-6689-4bee-aeeb-3aa8f556636a","Type":"ContainerDied","Data":"6856490c322842b453a7b2c22888abf32a0b5bbaac918d543c364e112d99e9b0"} Jul 02 17:31:07.801346 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.801256 2556 scope.go:117] "RemoveContainer" containerID="6d26715d501f46f6cbf8f03d98f965d02fd53b9517a1f9562be7cd8fd539e0bf" Jul 02 17:31:07.811692 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.811672 2556 scope.go:117] "RemoveContainer" containerID="f9ef308d7eedcd712690f9f3defb2fa0e492c81466071762a4a70bceb9bccfa2" Jul 02 17:31:07.819034 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.819012 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd"] Jul 02 17:31:07.820352 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.820332 2556 scope.go:117] "RemoveContainer" containerID="6d26715d501f46f6cbf8f03d98f965d02fd53b9517a1f9562be7cd8fd539e0bf" Jul 02 17:31:07.820632 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:31:07.820615 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d26715d501f46f6cbf8f03d98f965d02fd53b9517a1f9562be7cd8fd539e0bf\": container with ID starting with 6d26715d501f46f6cbf8f03d98f965d02fd53b9517a1f9562be7cd8fd539e0bf not found: ID does not exist" containerID="6d26715d501f46f6cbf8f03d98f965d02fd53b9517a1f9562be7cd8fd539e0bf" Jul 02 17:31:07.820713 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.820640 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d26715d501f46f6cbf8f03d98f965d02fd53b9517a1f9562be7cd8fd539e0bf"} err="failed to get container status \"6d26715d501f46f6cbf8f03d98f965d02fd53b9517a1f9562be7cd8fd539e0bf\": rpc error: code = NotFound desc = could not find container \"6d26715d501f46f6cbf8f03d98f965d02fd53b9517a1f9562be7cd8fd539e0bf\": container with ID starting with 6d26715d501f46f6cbf8f03d98f965d02fd53b9517a1f9562be7cd8fd539e0bf not found: ID does not exist" Jul 02 17:31:07.820713 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.820658 2556 scope.go:117] "RemoveContainer" containerID="f9ef308d7eedcd712690f9f3defb2fa0e492c81466071762a4a70bceb9bccfa2" Jul 02 17:31:07.820895 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:31:07.820881 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9ef308d7eedcd712690f9f3defb2fa0e492c81466071762a4a70bceb9bccfa2\": container with ID starting with f9ef308d7eedcd712690f9f3defb2fa0e492c81466071762a4a70bceb9bccfa2 not found: ID does not exist" containerID="f9ef308d7eedcd712690f9f3defb2fa0e492c81466071762a4a70bceb9bccfa2" Jul 02 17:31:07.820937 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.820901 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9ef308d7eedcd712690f9f3defb2fa0e492c81466071762a4a70bceb9bccfa2"} err="failed to get container status \"f9ef308d7eedcd712690f9f3defb2fa0e492c81466071762a4a70bceb9bccfa2\": rpc error: code = NotFound desc = could not find container \"f9ef308d7eedcd712690f9f3defb2fa0e492c81466071762a4a70bceb9bccfa2\": container with ID starting with f9ef308d7eedcd712690f9f3defb2fa0e492c81466071762a4a70bceb9bccfa2 not found: ID does not exist" Jul 02 17:31:07.824729 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:07.824709 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-50b097-openshift-default-f466ffff7-l42bd"] Jul 02 17:31:09.190121 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:09.190087 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc527878-6689-4bee-aeeb-3aa8f556636a" path="/var/lib/kubelet/pods/cc527878-6689-4bee-aeeb-3aa8f556636a/volumes" Jul 02 17:31:11.826220 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.826181 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc"] Jul 02 17:31:11.826845 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.826824 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc527878-6689-4bee-aeeb-3aa8f556636a" containerName="istio-proxy" Jul 02 17:31:11.826890 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.826850 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc527878-6689-4bee-aeeb-3aa8f556636a" containerName="istio-proxy" Jul 02 17:31:11.826890 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.826867 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc527878-6689-4bee-aeeb-3aa8f556636a" containerName="istio-proxy" Jul 02 17:31:11.826890 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.826876 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc527878-6689-4bee-aeeb-3aa8f556636a" containerName="istio-proxy" Jul 02 17:31:11.827022 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.827010 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc527878-6689-4bee-aeeb-3aa8f556636a" containerName="istio-proxy" Jul 02 17:31:11.827058 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.827025 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6116d61-38b8-429a-94d0-07b701ba1064" containerName="istio-proxy" Jul 02 17:31:11.832149 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.832128 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.834616 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.834591 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a295d5-openshift-default-dockercfg-xlbwh\"" Jul 02 17:31:11.844288 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.843146 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc"] Jul 02 17:31:11.867211 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.867136 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-workload-certs\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.867419 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.867249 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-workload-socket\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.867419 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.867277 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-data\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.867419 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.867303 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-podinfo\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.867419 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.867363 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-credential-socket\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.867419 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.867382 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istiod-ca-cert\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.867419 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.867406 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqkzf\" (UniqueName: \"kubernetes.io/projected/5f7702e4-91b1-4dc6-82a0-6901209c18e4-kube-api-access-hqkzf\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.867713 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.867437 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-token\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.867713 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.867525 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-envoy\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.968803 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.968767 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-workload-certs\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.968985 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.968815 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-workload-socket\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.968985 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.968845 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-data\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.968985 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.968878 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-podinfo\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.968985 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.968926 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-credential-socket\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.968985 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.968951 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istiod-ca-cert\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.968985 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.968974 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqkzf\" (UniqueName: \"kubernetes.io/projected/5f7702e4-91b1-4dc6-82a0-6901209c18e4-kube-api-access-hqkzf\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.969344 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.969029 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-token\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.969344 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.969098 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-envoy\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.969344 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.969275 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-workload-certs\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.969603 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.969344 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-workload-socket\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.969603 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.969371 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-data\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.969603 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.969560 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-credential-socket\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.969803 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.969779 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istiod-ca-cert\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.971272 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.971253 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-envoy\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.971500 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.971482 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-podinfo\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.977889 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.977868 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-token\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:11.978022 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:11.978005 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqkzf\" (UniqueName: \"kubernetes.io/projected/5f7702e4-91b1-4dc6-82a0-6901209c18e4-kube-api-access-hqkzf\") pod \"e2e-api-b-a295d5-openshift-default-db555449c-trlnc\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:12.151772 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:12.151693 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:12.292181 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:12.292140 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc"] Jul 02 17:31:12.294081 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:31:12.294054 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f7702e4_91b1_4dc6_82a0_6901209c18e4.slice/crio-f14251777c62a0843431cd9ebaf94337b2a07a8b820c19e279f721e9e655c584 WatchSource:0}: Error finding container f14251777c62a0843431cd9ebaf94337b2a07a8b820c19e279f721e9e655c584: Status 404 returned error can't find the container with id f14251777c62a0843431cd9ebaf94337b2a07a8b820c19e279f721e9e655c584 Jul 02 17:31:12.296800 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:12.296771 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:12.296882 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:12.296835 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:12.296882 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:12.296865 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:12.831087 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:12.831052 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" event={"ID":"5f7702e4-91b1-4dc6-82a0-6901209c18e4","Type":"ContainerStarted","Data":"e104ba5ebd0619d0510f2221a1f0adcc64c0ba00fcf1e64f70e4150b780576eb"} Jul 02 17:31:12.831508 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:12.831110 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" event={"ID":"5f7702e4-91b1-4dc6-82a0-6901209c18e4","Type":"ContainerStarted","Data":"f14251777c62a0843431cd9ebaf94337b2a07a8b820c19e279f721e9e655c584"} Jul 02 17:31:12.852652 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:12.852582 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" podStartSLOduration=1.852557992 podStartE2EDuration="1.852557992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:31:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:31:12.849656424 +0000 UTC m=+2206.162530093" watchObservedRunningTime="2026-07-02 17:31:12.852557992 +0000 UTC m=+2206.165431639" Jul 02 17:31:13.152730 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:13.152645 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:13.154280 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:13.154254 2556 patch_prober.go:28] interesting pod/e2e-api-b-a295d5-openshift-default-db555449c-trlnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 02 17:31:13.154402 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:13.154303 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" podUID="5f7702e4-91b1-4dc6-82a0-6901209c18e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 02 17:31:13.562235 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:13.562198 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:31:14.156599 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:14.156567 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:14.842220 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:14.842191 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:14.843251 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:14.843227 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:20.819318 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:20.819279 2556 patch_prober.go:28] interesting pod/e2e-api-a-28df41-openshift-default-6486665885-wrxpw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 17:31:20.819683 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:20.819363 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" podUID="34637434-e8b2-431a-b224-c2d5bd4816ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 17:31:21.878399 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:21.878368 2556 generic.go:356] "Generic (PLEG): container finished" podID="34637434-e8b2-431a-b224-c2d5bd4816ff" containerID="e37a830fbbe5fe9eefa038a29b81bea447d76c049de9a42352819ccc94b2dd6b" exitCode=0 Jul 02 17:31:21.878819 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:21.878438 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" event={"ID":"34637434-e8b2-431a-b224-c2d5bd4816ff","Type":"ContainerDied","Data":"e37a830fbbe5fe9eefa038a29b81bea447d76c049de9a42352819ccc94b2dd6b"} Jul 02 17:31:21.878865 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:21.878831 2556 scope.go:117] "RemoveContainer" containerID="e37a830fbbe5fe9eefa038a29b81bea447d76c049de9a42352819ccc94b2dd6b" Jul 02 17:31:21.879909 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:21.879882 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:21.879979 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:21.879953 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:21.880018 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:21.879991 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:22.886284 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:22.886250 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" event={"ID":"34637434-e8b2-431a-b224-c2d5bd4816ff","Type":"ContainerStarted","Data":"6621e300181315559fcbfdb997773d33be17d3b95edeca823f520842902230c9"} Jul 02 17:31:23.171270 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:23.171146 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:23.171423 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:23.171345 2556 patch_prober.go:28] interesting pod/e2e-api-a-28df41-openshift-default-6486665885-wrxpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 02 17:31:23.171423 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:23.171410 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" podUID="34637434-e8b2-431a-b224-c2d5bd4816ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 02 17:31:24.171395 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:24.171359 2556 patch_prober.go:28] interesting pod/e2e-api-a-28df41-openshift-default-6486665885-wrxpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 02 17:31:24.171870 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:24.171435 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" podUID="34637434-e8b2-431a-b224-c2d5bd4816ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 02 17:31:25.172031 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:25.171993 2556 patch_prober.go:28] interesting pod/e2e-api-a-28df41-openshift-default-6486665885-wrxpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 02 17:31:25.172594 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:25.172049 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" podUID="34637434-e8b2-431a-b224-c2d5bd4816ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 02 17:31:26.171762 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:26.171722 2556 patch_prober.go:28] interesting pod/e2e-api-a-28df41-openshift-default-6486665885-wrxpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 02 17:31:26.172256 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:26.171794 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" podUID="34637434-e8b2-431a-b224-c2d5bd4816ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 02 17:31:27.174578 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:27.174546 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:27.174972 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:27.174902 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:27.175517 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:27.175498 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:30.884346 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:30.884299 2556 patch_prober.go:28] interesting pod/e2e-api-b-a295d5-openshift-default-db555449c-trlnc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 17:31:30.884806 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:30.884363 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" podUID="5f7702e4-91b1-4dc6-82a0-6901209c18e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 17:31:31.564213 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:31.564179 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:31:31.940052 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:31.939951 2556 generic.go:356] "Generic (PLEG): container finished" podID="5f7702e4-91b1-4dc6-82a0-6901209c18e4" containerID="e104ba5ebd0619d0510f2221a1f0adcc64c0ba00fcf1e64f70e4150b780576eb" exitCode=0 Jul 02 17:31:31.940052 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:31.940025 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" event={"ID":"5f7702e4-91b1-4dc6-82a0-6901209c18e4","Type":"ContainerDied","Data":"e104ba5ebd0619d0510f2221a1f0adcc64c0ba00fcf1e64f70e4150b780576eb"} Jul 02 17:31:31.940514 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:31.940471 2556 scope.go:117] "RemoveContainer" containerID="e104ba5ebd0619d0510f2221a1f0adcc64c0ba00fcf1e64f70e4150b780576eb" Jul 02 17:31:31.941526 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:31.941499 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:31.941618 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:31.941564 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:31.941618 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:31.941596 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:32.947408 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:32.947374 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" event={"ID":"5f7702e4-91b1-4dc6-82a0-6901209c18e4","Type":"ContainerStarted","Data":"6ede1c78b1b02ac05331ff76f8e61fd23493f259a42b22382b0cddf857d9ab33"} Jul 02 17:31:33.152083 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:33.152043 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:33.152387 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:33.152363 2556 patch_prober.go:28] interesting pod/e2e-api-b-a295d5-openshift-default-db555449c-trlnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 02 17:31:33.152444 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:33.152418 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" podUID="5f7702e4-91b1-4dc6-82a0-6901209c18e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 02 17:31:34.152909 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:34.152875 2556 patch_prober.go:28] interesting pod/e2e-api-b-a295d5-openshift-default-db555449c-trlnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 02 17:31:34.153326 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:34.152929 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" podUID="5f7702e4-91b1-4dc6-82a0-6901209c18e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 02 17:31:35.020185 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:35.003870 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw"] Jul 02 17:31:35.020779 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:35.020738 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" podUID="34637434-e8b2-431a-b224-c2d5bd4816ff" containerName="istio-proxy" containerID="cri-o://6621e300181315559fcbfdb997773d33be17d3b95edeca823f520842902230c9" gracePeriod=30 Jul 02 17:31:35.152671 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:35.152632 2556 patch_prober.go:28] interesting pod/e2e-api-b-a295d5-openshift-default-db555449c-trlnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 02 17:31:35.152858 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:35.152700 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" podUID="5f7702e4-91b1-4dc6-82a0-6901209c18e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 02 17:31:36.152033 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:36.152000 2556 patch_prober.go:28] interesting pod/e2e-api-b-a295d5-openshift-default-db555449c-trlnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 02 17:31:36.152455 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:36.152063 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" podUID="5f7702e4-91b1-4dc6-82a0-6901209c18e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 02 17:31:37.155494 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:37.155461 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:37.155888 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:37.155858 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:37.156507 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:37.156487 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:40.275200 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.275178 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:40.367730 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.367636 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34637434-e8b2-431a-b224-c2d5bd4816ff-istiod-ca-cert\") pod \"34637434-e8b2-431a-b224-c2d5bd4816ff\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " Jul 02 17:31:40.367730 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.367683 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-token\") pod \"34637434-e8b2-431a-b224-c2d5bd4816ff\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " Jul 02 17:31:40.367730 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.367718 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-credential-socket\") pod \"34637434-e8b2-431a-b224-c2d5bd4816ff\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " Jul 02 17:31:40.368017 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.367815 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-workload-certs\") pod \"34637434-e8b2-431a-b224-c2d5bd4816ff\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " Jul 02 17:31:40.368017 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.367852 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-data\") pod \"34637434-e8b2-431a-b224-c2d5bd4816ff\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " Jul 02 17:31:40.368017 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.367903 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ck6qj\" (UniqueName: \"kubernetes.io/projected/34637434-e8b2-431a-b224-c2d5bd4816ff-kube-api-access-ck6qj\") pod \"34637434-e8b2-431a-b224-c2d5bd4816ff\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " Jul 02 17:31:40.368017 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.367931 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-podinfo\") pod \"34637434-e8b2-431a-b224-c2d5bd4816ff\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " Jul 02 17:31:40.368017 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.367964 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-envoy\") pod \"34637434-e8b2-431a-b224-c2d5bd4816ff\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " Jul 02 17:31:40.368017 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.367986 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-workload-socket\") pod \"34637434-e8b2-431a-b224-c2d5bd4816ff\" (UID: \"34637434-e8b2-431a-b224-c2d5bd4816ff\") " Jul 02 17:31:40.368366 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.368104 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "34637434-e8b2-431a-b224-c2d5bd4816ff" (UID: "34637434-e8b2-431a-b224-c2d5bd4816ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:40.368366 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.368009 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34637434-e8b2-431a-b224-c2d5bd4816ff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "34637434-e8b2-431a-b224-c2d5bd4816ff" (UID: "34637434-e8b2-431a-b224-c2d5bd4816ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:31:40.368366 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.368122 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "34637434-e8b2-431a-b224-c2d5bd4816ff" (UID: "34637434-e8b2-431a-b224-c2d5bd4816ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:40.368366 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.368340 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "34637434-e8b2-431a-b224-c2d5bd4816ff" (UID: "34637434-e8b2-431a-b224-c2d5bd4816ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:40.368580 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.368456 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-workload-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:40.368580 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.368479 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34637434-e8b2-431a-b224-c2d5bd4816ff-istiod-ca-cert\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:40.368580 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.368492 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-credential-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:40.368580 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.368506 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-workload-certs\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:40.368787 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.368684 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-data" (OuterVolumeSpecName: "istio-data") pod "34637434-e8b2-431a-b224-c2d5bd4816ff" (UID: "34637434-e8b2-431a-b224-c2d5bd4816ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:40.370039 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.370013 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-token" (OuterVolumeSpecName: "istio-token") pod "34637434-e8b2-431a-b224-c2d5bd4816ff" (UID: "34637434-e8b2-431a-b224-c2d5bd4816ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:31:40.370202 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.370173 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34637434-e8b2-431a-b224-c2d5bd4816ff-kube-api-access-ck6qj" (OuterVolumeSpecName: "kube-api-access-ck6qj") pod "34637434-e8b2-431a-b224-c2d5bd4816ff" (UID: "34637434-e8b2-431a-b224-c2d5bd4816ff"). InnerVolumeSpecName "kube-api-access-ck6qj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:31:40.370309 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.370243 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "34637434-e8b2-431a-b224-c2d5bd4816ff" (UID: "34637434-e8b2-431a-b224-c2d5bd4816ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:40.370353 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.370313 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "34637434-e8b2-431a-b224-c2d5bd4816ff" (UID: "34637434-e8b2-431a-b224-c2d5bd4816ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 17:31:40.470029 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.469990 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-data\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:40.470029 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.470027 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ck6qj\" (UniqueName: \"kubernetes.io/projected/34637434-e8b2-431a-b224-c2d5bd4816ff-kube-api-access-ck6qj\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:40.470029 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.470037 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-podinfo\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:40.470029 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.470047 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-envoy\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:40.470311 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.470055 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34637434-e8b2-431a-b224-c2d5bd4816ff-istio-token\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:40.991140 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.991099 2556 generic.go:356] "Generic (PLEG): container finished" podID="34637434-e8b2-431a-b224-c2d5bd4816ff" containerID="6621e300181315559fcbfdb997773d33be17d3b95edeca823f520842902230c9" exitCode=0 Jul 02 17:31:40.991347 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.991150 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" Jul 02 17:31:40.991347 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.991192 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" event={"ID":"34637434-e8b2-431a-b224-c2d5bd4816ff","Type":"ContainerDied","Data":"6621e300181315559fcbfdb997773d33be17d3b95edeca823f520842902230c9"} Jul 02 17:31:40.991347 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.991248 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw" event={"ID":"34637434-e8b2-431a-b224-c2d5bd4816ff","Type":"ContainerDied","Data":"231229c6f324085b35fe7084c1f3cd7f3bf25b34d3da1938bdce477a79c31b6e"} Jul 02 17:31:40.991347 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:40.991289 2556 scope.go:117] "RemoveContainer" containerID="6621e300181315559fcbfdb997773d33be17d3b95edeca823f520842902230c9" Jul 02 17:31:41.000941 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:41.000915 2556 scope.go:117] "RemoveContainer" containerID="e37a830fbbe5fe9eefa038a29b81bea447d76c049de9a42352819ccc94b2dd6b" Jul 02 17:31:41.009865 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:41.009846 2556 scope.go:117] "RemoveContainer" containerID="6621e300181315559fcbfdb997773d33be17d3b95edeca823f520842902230c9" Jul 02 17:31:41.010147 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:31:41.010128 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6621e300181315559fcbfdb997773d33be17d3b95edeca823f520842902230c9\": container with ID starting with 6621e300181315559fcbfdb997773d33be17d3b95edeca823f520842902230c9 not found: ID does not exist" containerID="6621e300181315559fcbfdb997773d33be17d3b95edeca823f520842902230c9" Jul 02 17:31:41.010211 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:41.010177 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6621e300181315559fcbfdb997773d33be17d3b95edeca823f520842902230c9"} err="failed to get container status \"6621e300181315559fcbfdb997773d33be17d3b95edeca823f520842902230c9\": rpc error: code = NotFound desc = could not find container \"6621e300181315559fcbfdb997773d33be17d3b95edeca823f520842902230c9\": container with ID starting with 6621e300181315559fcbfdb997773d33be17d3b95edeca823f520842902230c9 not found: ID does not exist" Jul 02 17:31:41.010211 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:41.010199 2556 scope.go:117] "RemoveContainer" containerID="e37a830fbbe5fe9eefa038a29b81bea447d76c049de9a42352819ccc94b2dd6b" Jul 02 17:31:41.010468 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:31:41.010450 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e37a830fbbe5fe9eefa038a29b81bea447d76c049de9a42352819ccc94b2dd6b\": container with ID starting with e37a830fbbe5fe9eefa038a29b81bea447d76c049de9a42352819ccc94b2dd6b not found: ID does not exist" containerID="e37a830fbbe5fe9eefa038a29b81bea447d76c049de9a42352819ccc94b2dd6b" Jul 02 17:31:41.010534 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:41.010473 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e37a830fbbe5fe9eefa038a29b81bea447d76c049de9a42352819ccc94b2dd6b"} err="failed to get container status \"e37a830fbbe5fe9eefa038a29b81bea447d76c049de9a42352819ccc94b2dd6b\": rpc error: code = NotFound desc = could not find container \"e37a830fbbe5fe9eefa038a29b81bea447d76c049de9a42352819ccc94b2dd6b\": container with ID starting with e37a830fbbe5fe9eefa038a29b81bea447d76c049de9a42352819ccc94b2dd6b not found: ID does not exist" Jul 02 17:31:41.016438 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:41.016418 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw"] Jul 02 17:31:41.024803 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:41.024781 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-28df41-openshift-default-6486665885-wrxpw"] Jul 02 17:31:41.190465 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:41.190436 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34637434-e8b2-431a-b224-c2d5bd4816ff" path="/var/lib/kubelet/pods/34637434-e8b2-431a-b224-c2d5bd4816ff/volumes" Jul 02 17:31:41.824240 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:41.824199 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc"] Jul 02 17:31:41.824692 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:41.824441 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" podUID="5f7702e4-91b1-4dc6-82a0-6901209c18e4" containerName="istio-proxy" containerID="cri-o://6ede1c78b1b02ac05331ff76f8e61fd23493f259a42b22382b0cddf857d9ab33" gracePeriod=30 Jul 02 17:31:42.709856 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.709826 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7"] Jul 02 17:31:42.710267 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.710252 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34637434-e8b2-431a-b224-c2d5bd4816ff" containerName="istio-proxy" Jul 02 17:31:42.710356 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.710268 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="34637434-e8b2-431a-b224-c2d5bd4816ff" containerName="istio-proxy" Jul 02 17:31:42.710356 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.710286 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34637434-e8b2-431a-b224-c2d5bd4816ff" containerName="istio-proxy" Jul 02 17:31:42.710356 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.710292 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="34637434-e8b2-431a-b224-c2d5bd4816ff" containerName="istio-proxy" Jul 02 17:31:42.710356 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.710352 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="34637434-e8b2-431a-b224-c2d5bd4816ff" containerName="istio-proxy" Jul 02 17:31:42.710557 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.710361 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="34637434-e8b2-431a-b224-c2d5bd4816ff" containerName="istio-proxy" Jul 02 17:31:42.710557 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.710373 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc527878-6689-4bee-aeeb-3aa8f556636a" containerName="istio-proxy" Jul 02 17:31:42.713835 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.713815 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.716054 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.716034 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a0a2ac-openshift-default-dockercfg-5p62n\"" Jul 02 17:31:42.723780 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.723756 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7"] Jul 02 17:31:42.895414 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.895380 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-podinfo\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.895786 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.895420 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-credential-socket\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.895786 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.895519 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-workload-certs\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.895786 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.895555 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-token\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.895786 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.895583 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-data\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.895786 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.895662 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hdtw\" (UniqueName: \"kubernetes.io/projected/08d4f453-f5fa-4860-beb9-2f0820144eec-kube-api-access-7hdtw\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.895786 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.895697 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-envoy\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.895786 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.895718 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08d4f453-f5fa-4860-beb9-2f0820144eec-istiod-ca-cert\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.895786 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.895739 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-workload-socket\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.996807 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.996725 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-workload-certs\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.996807 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.996758 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-token\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.996807 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.996788 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-data\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.997073 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.996829 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hdtw\" (UniqueName: \"kubernetes.io/projected/08d4f453-f5fa-4860-beb9-2f0820144eec-kube-api-access-7hdtw\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.997073 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.996856 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-envoy\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.997073 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.997013 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08d4f453-f5fa-4860-beb9-2f0820144eec-istiod-ca-cert\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.997073 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.997070 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-workload-socket\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.997311 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.997138 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-podinfo\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.997311 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.997208 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-credential-socket\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.997399 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.997324 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-data\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.997399 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.997389 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-workload-certs\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.997493 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.997431 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-workload-socket\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.997652 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.997630 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-credential-socket\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.997868 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.997816 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08d4f453-f5fa-4860-beb9-2f0820144eec-istiod-ca-cert\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.999375 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.999351 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-envoy\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:42.999865 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:42.999845 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-podinfo\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:43.005066 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:43.005037 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-token\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:43.005471 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:43.005450 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hdtw\" (UniqueName: \"kubernetes.io/projected/08d4f453-f5fa-4860-beb9-2f0820144eec-kube-api-access-7hdtw\") pod \"e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:43.025295 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:43.025274 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:43.162680 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:43.162655 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7"] Jul 02 17:31:43.164811 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:31:43.164784 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d4f453_f5fa_4860_beb9_2f0820144eec.slice/crio-bd2c780f90c048e3f4b7c088674c2973795fda4470911ff6f33f64c4b787158b WatchSource:0}: Error finding container bd2c780f90c048e3f4b7c088674c2973795fda4470911ff6f33f64c4b787158b: Status 404 returned error can't find the container with id bd2c780f90c048e3f4b7c088674c2973795fda4470911ff6f33f64c4b787158b Jul 02 17:31:43.167512 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:43.167476 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:43.167602 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:43.167546 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:43.167602 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:43.167579 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:44.011964 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:44.011925 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" event={"ID":"08d4f453-f5fa-4860-beb9-2f0820144eec","Type":"ContainerStarted","Data":"aa751abbd9da74bc7b8928a22d6b1b32cf3bd776abcc8d74bc699bbbbb3a2c3b"} Jul 02 17:31:44.012428 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:44.011982 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" event={"ID":"08d4f453-f5fa-4860-beb9-2f0820144eec","Type":"ContainerStarted","Data":"bd2c780f90c048e3f4b7c088674c2973795fda4470911ff6f33f64c4b787158b"} Jul 02 17:31:44.026223 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:44.026199 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:44.031830 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:44.031806 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:44.031967 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:44.031930 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" podStartSLOduration=2.031915232 podStartE2EDuration="2.031915232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:31:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:31:44.02967327 +0000 UTC m=+2237.342546951" watchObservedRunningTime="2026-07-02 17:31:44.031915232 +0000 UTC m=+2237.344788877" Jul 02 17:31:45.017537 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:45.017510 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:45.018769 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:45.018742 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:31:45.962428 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:45.962384 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:31:47.031321 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.031291 2556 generic.go:356] "Generic (PLEG): container finished" podID="5f7702e4-91b1-4dc6-82a0-6901209c18e4" containerID="6ede1c78b1b02ac05331ff76f8e61fd23493f259a42b22382b0cddf857d9ab33" exitCode=0 Jul 02 17:31:47.031699 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.031365 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" event={"ID":"5f7702e4-91b1-4dc6-82a0-6901209c18e4","Type":"ContainerDied","Data":"6ede1c78b1b02ac05331ff76f8e61fd23493f259a42b22382b0cddf857d9ab33"} Jul 02 17:31:47.031699 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.031433 2556 scope.go:117] "RemoveContainer" containerID="e104ba5ebd0619d0510f2221a1f0adcc64c0ba00fcf1e64f70e4150b780576eb" Jul 02 17:31:47.074328 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.074306 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:47.132995 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.132922 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqkzf\" (UniqueName: \"kubernetes.io/projected/5f7702e4-91b1-4dc6-82a0-6901209c18e4-kube-api-access-hqkzf\") pod \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " Jul 02 17:31:47.132995 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.132963 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-envoy\") pod \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " Jul 02 17:31:47.133223 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.133025 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-workload-certs\") pod \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " Jul 02 17:31:47.133223 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.133044 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-podinfo\") pod \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " Jul 02 17:31:47.133223 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.133062 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-data\") pod \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " Jul 02 17:31:47.133223 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.133081 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istiod-ca-cert\") pod \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " Jul 02 17:31:47.133223 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.133119 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-credential-socket\") pod \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " Jul 02 17:31:47.133223 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.133223 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-workload-socket\") pod \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " Jul 02 17:31:47.133689 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.133246 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-token\") pod \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\" (UID: \"5f7702e4-91b1-4dc6-82a0-6901209c18e4\") " Jul 02 17:31:47.133689 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.133335 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f7702e4-91b1-4dc6-82a0-6901209c18e4" (UID: "5f7702e4-91b1-4dc6-82a0-6901209c18e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:47.133689 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.133502 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f7702e4-91b1-4dc6-82a0-6901209c18e4" (UID: "5f7702e4-91b1-4dc6-82a0-6901209c18e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:47.133689 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.133552 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f7702e4-91b1-4dc6-82a0-6901209c18e4" (UID: "5f7702e4-91b1-4dc6-82a0-6901209c18e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:31:47.133689 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.133635 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-workload-certs\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:47.133689 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.133654 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istiod-ca-cert\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:47.133689 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.133669 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-workload-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:47.134190 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.133914 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f7702e4-91b1-4dc6-82a0-6901209c18e4" (UID: "5f7702e4-91b1-4dc6-82a0-6901209c18e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:47.134190 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.134024 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f7702e4-91b1-4dc6-82a0-6901209c18e4" (UID: "5f7702e4-91b1-4dc6-82a0-6901209c18e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:47.135328 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.135301 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f7702e4-91b1-4dc6-82a0-6901209c18e4" (UID: "5f7702e4-91b1-4dc6-82a0-6901209c18e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:31:47.135417 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.135332 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f7702e4-91b1-4dc6-82a0-6901209c18e4-kube-api-access-hqkzf" (OuterVolumeSpecName: "kube-api-access-hqkzf") pod "5f7702e4-91b1-4dc6-82a0-6901209c18e4" (UID: "5f7702e4-91b1-4dc6-82a0-6901209c18e4"). InnerVolumeSpecName "kube-api-access-hqkzf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:31:47.135417 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.135380 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f7702e4-91b1-4dc6-82a0-6901209c18e4" (UID: "5f7702e4-91b1-4dc6-82a0-6901209c18e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 17:31:47.135874 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.135859 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f7702e4-91b1-4dc6-82a0-6901209c18e4" (UID: "5f7702e4-91b1-4dc6-82a0-6901209c18e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:31:47.235074 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.235044 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-credential-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:47.235074 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.235069 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-token\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:47.235074 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.235080 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqkzf\" (UniqueName: \"kubernetes.io/projected/5f7702e4-91b1-4dc6-82a0-6901209c18e4-kube-api-access-hqkzf\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:47.235294 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.235089 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-envoy\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:47.235294 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.235098 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-podinfo\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:47.235294 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:47.235105 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f7702e4-91b1-4dc6-82a0-6901209c18e4-istio-data\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:31:48.037719 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:48.037691 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" Jul 02 17:31:48.037719 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:48.037700 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc" event={"ID":"5f7702e4-91b1-4dc6-82a0-6901209c18e4","Type":"ContainerDied","Data":"f14251777c62a0843431cd9ebaf94337b2a07a8b820c19e279f721e9e655c584"} Jul 02 17:31:48.038278 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:48.037760 2556 scope.go:117] "RemoveContainer" containerID="6ede1c78b1b02ac05331ff76f8e61fd23493f259a42b22382b0cddf857d9ab33" Jul 02 17:31:48.056536 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:48.056507 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc"] Jul 02 17:31:48.060403 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:48.060369 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a295d5-openshift-default-db555449c-trlnc"] Jul 02 17:31:49.190529 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:49.190445 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f7702e4-91b1-4dc6-82a0-6901209c18e4" path="/var/lib/kubelet/pods/5f7702e4-91b1-4dc6-82a0-6901209c18e4/volumes" Jul 02 17:31:51.167641 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.167608 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk"] Jul 02 17:31:51.168226 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.168208 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f7702e4-91b1-4dc6-82a0-6901209c18e4" containerName="istio-proxy" Jul 02 17:31:51.168347 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.168228 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f7702e4-91b1-4dc6-82a0-6901209c18e4" containerName="istio-proxy" Jul 02 17:31:51.168347 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.168247 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f7702e4-91b1-4dc6-82a0-6901209c18e4" containerName="istio-proxy" Jul 02 17:31:51.168347 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.168255 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f7702e4-91b1-4dc6-82a0-6901209c18e4" containerName="istio-proxy" Jul 02 17:31:51.168496 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.168356 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f7702e4-91b1-4dc6-82a0-6901209c18e4" containerName="istio-proxy" Jul 02 17:31:51.172654 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.172632 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.175322 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.175300 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-4421aa-openshift-default-dockercfg-dskdz\"" Jul 02 17:31:51.196479 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.196450 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk"] Jul 02 17:31:51.274020 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.273977 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-workload-certs\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.274215 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.274045 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8856a07-7f1e-4359-845f-e041ca002854-istio-token\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.274273 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.274213 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8856a07-7f1e-4359-845f-e041ca002854-istio-podinfo\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.274273 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.274264 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-istio-data\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.274359 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.274314 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-workload-socket\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.274408 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.274369 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-istio-envoy\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.274408 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.274397 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8856a07-7f1e-4359-845f-e041ca002854-istiod-ca-cert\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.274479 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.274420 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqf4d\" (UniqueName: \"kubernetes.io/projected/a8856a07-7f1e-4359-845f-e041ca002854-kube-api-access-kqf4d\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.274479 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.274452 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-credential-socket\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.375046 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.375007 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-istio-envoy\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.375046 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.375045 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8856a07-7f1e-4359-845f-e041ca002854-istiod-ca-cert\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.375335 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.375065 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqf4d\" (UniqueName: \"kubernetes.io/projected/a8856a07-7f1e-4359-845f-e041ca002854-kube-api-access-kqf4d\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.375335 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.375092 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-credential-socket\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.375335 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.375119 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-workload-certs\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.375335 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.375154 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8856a07-7f1e-4359-845f-e041ca002854-istio-token\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.375335 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.375271 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8856a07-7f1e-4359-845f-e041ca002854-istio-podinfo\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.375335 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.375308 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-istio-data\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.375686 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.375350 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-workload-socket\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.375686 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.375606 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-workload-certs\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.375686 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.375629 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-credential-socket\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.375799 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.375685 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-workload-socket\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.375799 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.375724 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-istio-data\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.376101 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.376082 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8856a07-7f1e-4359-845f-e041ca002854-istiod-ca-cert\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.377376 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.377355 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-istio-envoy\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.377602 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.377581 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8856a07-7f1e-4359-845f-e041ca002854-istio-podinfo\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.385771 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.385745 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8856a07-7f1e-4359-845f-e041ca002854-istio-token\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.385917 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.385897 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqf4d\" (UniqueName: \"kubernetes.io/projected/a8856a07-7f1e-4359-845f-e041ca002854-kube-api-access-kqf4d\") pod \"e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.483236 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.483128 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:51.635842 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.635816 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk"] Jul 02 17:31:51.637211 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:31:51.637188 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8856a07_7f1e_4359_845f_e041ca002854.slice/crio-f0900348eb05463660eb979ad6abc7d5b818d9d0cc05fd42a51418c501c3e800 WatchSource:0}: Error finding container f0900348eb05463660eb979ad6abc7d5b818d9d0cc05fd42a51418c501c3e800: Status 404 returned error can't find the container with id f0900348eb05463660eb979ad6abc7d5b818d9d0cc05fd42a51418c501c3e800 Jul 02 17:31:51.639782 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.639746 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:51.639888 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.639825 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:51.639888 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:51.639870 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:31:52.060765 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:52.060727 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" event={"ID":"a8856a07-7f1e-4359-845f-e041ca002854","Type":"ContainerStarted","Data":"e0cc4ebe1f76a16a6ac63f2290f7f1a82a3e7fea5e063c7730a4dbd17317ac7c"} Jul 02 17:31:52.060975 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:52.060785 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" event={"ID":"a8856a07-7f1e-4359-845f-e041ca002854","Type":"ContainerStarted","Data":"f0900348eb05463660eb979ad6abc7d5b818d9d0cc05fd42a51418c501c3e800"} Jul 02 17:31:52.082041 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:52.081991 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" podStartSLOduration=1.081973795 podStartE2EDuration="1.081973795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:31:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:31:52.079351633 +0000 UTC m=+2245.392225278" watchObservedRunningTime="2026-07-02 17:31:52.081973795 +0000 UTC m=+2245.394847501" Jul 02 17:31:52.483474 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:52.483368 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:52.488991 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:52.488959 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:53.067796 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:53.067765 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:53.068725 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:53.068709 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:31:58.367098 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:31:58.367065 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:32:00.019243 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:00.019205 2556 patch_prober.go:28] interesting pod/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 02 17:32:00.020189 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:00.019856 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" podUID="08d4f453-f5fa-4860-beb9-2f0820144eec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 02 17:32:00.111900 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:00.111864 2556 generic.go:356] "Generic (PLEG): container finished" podID="08d4f453-f5fa-4860-beb9-2f0820144eec" containerID="aa751abbd9da74bc7b8928a22d6b1b32cf3bd776abcc8d74bc699bbbbb3a2c3b" exitCode=0 Jul 02 17:32:00.112106 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:00.111952 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" event={"ID":"08d4f453-f5fa-4860-beb9-2f0820144eec","Type":"ContainerDied","Data":"aa751abbd9da74bc7b8928a22d6b1b32cf3bd776abcc8d74bc699bbbbb3a2c3b"} Jul 02 17:32:00.112844 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:00.112817 2556 scope.go:117] "RemoveContainer" containerID="aa751abbd9da74bc7b8928a22d6b1b32cf3bd776abcc8d74bc699bbbbb3a2c3b" Jul 02 17:32:00.114907 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:00.114137 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:32:00.114907 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:00.114245 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:32:00.114907 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:00.114295 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:32:01.123400 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:01.123359 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" event={"ID":"08d4f453-f5fa-4860-beb9-2f0820144eec","Type":"ContainerStarted","Data":"e8abcf1bf55d945adb4438478a48f55fd4a0825f2d3a4d35d140a2a66ec9648d"} Jul 02 17:32:02.025832 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:02.025791 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:32:02.026310 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:02.026280 2556 patch_prober.go:28] interesting pod/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 02 17:32:02.026555 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:02.026527 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" podUID="08d4f453-f5fa-4860-beb9-2f0820144eec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 02 17:32:03.026781 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:03.026740 2556 patch_prober.go:28] interesting pod/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 02 17:32:03.027890 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:03.027411 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" podUID="08d4f453-f5fa-4860-beb9-2f0820144eec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 02 17:32:04.026934 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:04.026452 2556 patch_prober.go:28] interesting pod/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 02 17:32:04.026934 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:04.026540 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" podUID="08d4f453-f5fa-4860-beb9-2f0820144eec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 02 17:32:05.026876 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:05.026383 2556 patch_prober.go:28] interesting pod/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 02 17:32:05.026876 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:05.026475 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" podUID="08d4f453-f5fa-4860-beb9-2f0820144eec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 02 17:32:06.030486 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:06.030454 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:32:06.030946 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:06.030736 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:32:06.031599 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:06.031581 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:32:07.165184 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:07.165139 2556 generic.go:356] "Generic (PLEG): container finished" podID="a8856a07-7f1e-4359-845f-e041ca002854" containerID="e0cc4ebe1f76a16a6ac63f2290f7f1a82a3e7fea5e063c7730a4dbd17317ac7c" exitCode=0 Jul 02 17:32:07.165694 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:07.165214 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" event={"ID":"a8856a07-7f1e-4359-845f-e041ca002854","Type":"ContainerDied","Data":"e0cc4ebe1f76a16a6ac63f2290f7f1a82a3e7fea5e063c7730a4dbd17317ac7c"} Jul 02 17:32:07.165768 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:07.165740 2556 scope.go:117] "RemoveContainer" containerID="e0cc4ebe1f76a16a6ac63f2290f7f1a82a3e7fea5e063c7730a4dbd17317ac7c" Jul 02 17:32:07.166740 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:07.166713 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:32:07.166855 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:07.166839 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:32:07.166922 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:07.166899 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 02 17:32:08.068265 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:08.068229 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:32:08.172968 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:08.172929 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" event={"ID":"a8856a07-7f1e-4359-845f-e041ca002854","Type":"ContainerStarted","Data":"36c1265b73ac2e7b34fd34b3a4ab48fabe01ba06d7517313b726030920477456"} Jul 02 17:32:08.483529 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:08.483434 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:32:08.484728 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:08.484698 2556 patch_prober.go:28] interesting pod/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 02 17:32:08.484871 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:08.484788 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" podUID="a8856a07-7f1e-4359-845f-e041ca002854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 02 17:32:09.484256 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:09.484212 2556 patch_prober.go:28] interesting pod/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 02 17:32:09.484685 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:09.484289 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" podUID="a8856a07-7f1e-4359-845f-e041ca002854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 02 17:32:10.483907 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:10.483869 2556 patch_prober.go:28] interesting pod/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 02 17:32:10.484102 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:10.483932 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" podUID="a8856a07-7f1e-4359-845f-e041ca002854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 02 17:32:11.484054 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:11.484017 2556 patch_prober.go:28] interesting pod/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 02 17:32:11.484479 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:11.484087 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" podUID="a8856a07-7f1e-4359-845f-e041ca002854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 02 17:32:12.487496 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:12.487459 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:32:12.487961 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:12.487758 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:32:12.488547 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:12.488530 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:32:16.960979 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:16.960941 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:32:37.062694 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:37.062659 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:32:48.153874 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:32:48.153840 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:33:04.264052 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:33:04.264018 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:33:19.461003 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:33:19.460919 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:33:35.561980 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:33:35.561946 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:33:51.662265 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:33:51.662226 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:34:06.563378 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:34:06.563337 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:34:24.055654 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:34:24.055620 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:34:27.366273 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:34:27.366244 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 17:34:27.371784 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:34:27.371763 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 17:34:40.676388 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:34:40.676354 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:35:08.563994 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:35:08.563912 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:35:32.161485 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:35:32.161448 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:35:52.061954 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:35:52.061913 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:36:09.855315 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:36:09.855275 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:36:37.870628 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:36:37.870598 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:37:05.560789 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:37:05.560750 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:37:19.959383 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:37:19.959348 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:37:33.160858 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:37:33.160826 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:38:10.713375 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:10.713295 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7"] Jul 02 17:38:10.713938 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:10.713568 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" podUID="08d4f453-f5fa-4860-beb9-2f0820144eec" containerName="istio-proxy" containerID="cri-o://e8abcf1bf55d945adb4438478a48f55fd4a0825f2d3a4d35d140a2a66ec9648d" gracePeriod=30 Jul 02 17:38:14.662145 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:14.662110 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gn5fl"] Jul 02 17:38:15.978979 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:15.978923 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:38:16.063369 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.063326 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-workload-socket\") pod \"08d4f453-f5fa-4860-beb9-2f0820144eec\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " Jul 02 17:38:16.063555 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.063398 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-token\") pod \"08d4f453-f5fa-4860-beb9-2f0820144eec\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " Jul 02 17:38:16.063555 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.063419 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-envoy\") pod \"08d4f453-f5fa-4860-beb9-2f0820144eec\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " Jul 02 17:38:16.063555 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.063442 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08d4f453-f5fa-4860-beb9-2f0820144eec-istiod-ca-cert\") pod \"08d4f453-f5fa-4860-beb9-2f0820144eec\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " Jul 02 17:38:16.063555 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.063462 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hdtw\" (UniqueName: \"kubernetes.io/projected/08d4f453-f5fa-4860-beb9-2f0820144eec-kube-api-access-7hdtw\") pod \"08d4f453-f5fa-4860-beb9-2f0820144eec\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " Jul 02 17:38:16.063555 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.063489 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-credential-socket\") pod \"08d4f453-f5fa-4860-beb9-2f0820144eec\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " Jul 02 17:38:16.063555 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.063503 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-data\") pod \"08d4f453-f5fa-4860-beb9-2f0820144eec\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " Jul 02 17:38:16.063555 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.063536 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-podinfo\") pod \"08d4f453-f5fa-4860-beb9-2f0820144eec\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " Jul 02 17:38:16.063931 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.063568 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-workload-certs\") pod \"08d4f453-f5fa-4860-beb9-2f0820144eec\" (UID: \"08d4f453-f5fa-4860-beb9-2f0820144eec\") " Jul 02 17:38:16.063931 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.063664 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "08d4f453-f5fa-4860-beb9-2f0820144eec" (UID: "08d4f453-f5fa-4860-beb9-2f0820144eec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:38:16.064050 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.063937 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-workload-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:16.064050 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.063972 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08d4f453-f5fa-4860-beb9-2f0820144eec-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "08d4f453-f5fa-4860-beb9-2f0820144eec" (UID: "08d4f453-f5fa-4860-beb9-2f0820144eec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:38:16.064210 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.064144 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "08d4f453-f5fa-4860-beb9-2f0820144eec" (UID: "08d4f453-f5fa-4860-beb9-2f0820144eec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:38:16.064346 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.064217 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "08d4f453-f5fa-4860-beb9-2f0820144eec" (UID: "08d4f453-f5fa-4860-beb9-2f0820144eec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:38:16.064573 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.064547 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-data" (OuterVolumeSpecName: "istio-data") pod "08d4f453-f5fa-4860-beb9-2f0820144eec" (UID: "08d4f453-f5fa-4860-beb9-2f0820144eec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:38:16.065782 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.065759 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "08d4f453-f5fa-4860-beb9-2f0820144eec" (UID: "08d4f453-f5fa-4860-beb9-2f0820144eec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:38:16.065782 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.065778 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-token" (OuterVolumeSpecName: "istio-token") pod "08d4f453-f5fa-4860-beb9-2f0820144eec" (UID: "08d4f453-f5fa-4860-beb9-2f0820144eec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:38:16.065921 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.065759 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08d4f453-f5fa-4860-beb9-2f0820144eec-kube-api-access-7hdtw" (OuterVolumeSpecName: "kube-api-access-7hdtw") pod "08d4f453-f5fa-4860-beb9-2f0820144eec" (UID: "08d4f453-f5fa-4860-beb9-2f0820144eec"). InnerVolumeSpecName "kube-api-access-7hdtw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:38:16.065921 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.065903 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "08d4f453-f5fa-4860-beb9-2f0820144eec" (UID: "08d4f453-f5fa-4860-beb9-2f0820144eec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 17:38:16.100680 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.100648 2556 generic.go:356] "Generic (PLEG): container finished" podID="08d4f453-f5fa-4860-beb9-2f0820144eec" containerID="e8abcf1bf55d945adb4438478a48f55fd4a0825f2d3a4d35d140a2a66ec9648d" exitCode=0 Jul 02 17:38:16.100863 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.100697 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" Jul 02 17:38:16.100863 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.100726 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" event={"ID":"08d4f453-f5fa-4860-beb9-2f0820144eec","Type":"ContainerDied","Data":"e8abcf1bf55d945adb4438478a48f55fd4a0825f2d3a4d35d140a2a66ec9648d"} Jul 02 17:38:16.100863 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.100787 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7" event={"ID":"08d4f453-f5fa-4860-beb9-2f0820144eec","Type":"ContainerDied","Data":"bd2c780f90c048e3f4b7c088674c2973795fda4470911ff6f33f64c4b787158b"} Jul 02 17:38:16.100863 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.100819 2556 scope.go:117] "RemoveContainer" containerID="e8abcf1bf55d945adb4438478a48f55fd4a0825f2d3a4d35d140a2a66ec9648d" Jul 02 17:38:16.112937 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.112918 2556 scope.go:117] "RemoveContainer" containerID="aa751abbd9da74bc7b8928a22d6b1b32cf3bd776abcc8d74bc699bbbbb3a2c3b" Jul 02 17:38:16.121285 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.121267 2556 scope.go:117] "RemoveContainer" containerID="e8abcf1bf55d945adb4438478a48f55fd4a0825f2d3a4d35d140a2a66ec9648d" Jul 02 17:38:16.121533 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:38:16.121516 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8abcf1bf55d945adb4438478a48f55fd4a0825f2d3a4d35d140a2a66ec9648d\": container with ID starting with e8abcf1bf55d945adb4438478a48f55fd4a0825f2d3a4d35d140a2a66ec9648d not found: ID does not exist" containerID="e8abcf1bf55d945adb4438478a48f55fd4a0825f2d3a4d35d140a2a66ec9648d" Jul 02 17:38:16.121598 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.121542 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8abcf1bf55d945adb4438478a48f55fd4a0825f2d3a4d35d140a2a66ec9648d"} err="failed to get container status \"e8abcf1bf55d945adb4438478a48f55fd4a0825f2d3a4d35d140a2a66ec9648d\": rpc error: code = NotFound desc = could not find container \"e8abcf1bf55d945adb4438478a48f55fd4a0825f2d3a4d35d140a2a66ec9648d\": container with ID starting with e8abcf1bf55d945adb4438478a48f55fd4a0825f2d3a4d35d140a2a66ec9648d not found: ID does not exist" Jul 02 17:38:16.121598 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.121559 2556 scope.go:117] "RemoveContainer" containerID="aa751abbd9da74bc7b8928a22d6b1b32cf3bd776abcc8d74bc699bbbbb3a2c3b" Jul 02 17:38:16.121779 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:38:16.121763 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa751abbd9da74bc7b8928a22d6b1b32cf3bd776abcc8d74bc699bbbbb3a2c3b\": container with ID starting with aa751abbd9da74bc7b8928a22d6b1b32cf3bd776abcc8d74bc699bbbbb3a2c3b not found: ID does not exist" containerID="aa751abbd9da74bc7b8928a22d6b1b32cf3bd776abcc8d74bc699bbbbb3a2c3b" Jul 02 17:38:16.121836 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.121782 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa751abbd9da74bc7b8928a22d6b1b32cf3bd776abcc8d74bc699bbbbb3a2c3b"} err="failed to get container status \"aa751abbd9da74bc7b8928a22d6b1b32cf3bd776abcc8d74bc699bbbbb3a2c3b\": rpc error: code = NotFound desc = could not find container \"aa751abbd9da74bc7b8928a22d6b1b32cf3bd776abcc8d74bc699bbbbb3a2c3b\": container with ID starting with aa751abbd9da74bc7b8928a22d6b1b32cf3bd776abcc8d74bc699bbbbb3a2c3b not found: ID does not exist" Jul 02 17:38:16.122985 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.122964 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7"] Jul 02 17:38:16.126303 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.126284 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a0a2ac-openshift-default-5f9c678b4c-nwdq7"] Jul 02 17:38:16.165417 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.165380 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-credential-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:16.165417 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.165415 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-data\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:16.165417 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.165427 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-podinfo\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:16.165663 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.165435 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-workload-certs\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:16.165663 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.165446 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-token\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:16.165663 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.165453 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08d4f453-f5fa-4860-beb9-2f0820144eec-istio-envoy\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:16.165663 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.165460 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08d4f453-f5fa-4860-beb9-2f0820144eec-istiod-ca-cert\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:16.165663 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:16.165468 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hdtw\" (UniqueName: \"kubernetes.io/projected/08d4f453-f5fa-4860-beb9-2f0820144eec-kube-api-access-7hdtw\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:17.189689 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:17.189658 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08d4f453-f5fa-4860-beb9-2f0820144eec" path="/var/lib/kubelet/pods/08d4f453-f5fa-4860-beb9-2f0820144eec/volumes" Jul 02 17:38:17.871919 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:17.871886 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk"] Jul 02 17:38:17.872104 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:17.872087 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" podUID="a8856a07-7f1e-4359-845f-e041ca002854" containerName="istio-proxy" containerID="cri-o://36c1265b73ac2e7b34fd34b3a4ab48fabe01ba06d7517313b726030920477456" gracePeriod=30 Jul 02 17:38:23.117698 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.117676 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:38:23.141931 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.141858 2556 generic.go:356] "Generic (PLEG): container finished" podID="a8856a07-7f1e-4359-845f-e041ca002854" containerID="36c1265b73ac2e7b34fd34b3a4ab48fabe01ba06d7517313b726030920477456" exitCode=0 Jul 02 17:38:23.142063 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.141929 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" Jul 02 17:38:23.142063 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.141941 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" event={"ID":"a8856a07-7f1e-4359-845f-e041ca002854","Type":"ContainerDied","Data":"36c1265b73ac2e7b34fd34b3a4ab48fabe01ba06d7517313b726030920477456"} Jul 02 17:38:23.142063 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.142025 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk" event={"ID":"a8856a07-7f1e-4359-845f-e041ca002854","Type":"ContainerDied","Data":"f0900348eb05463660eb979ad6abc7d5b818d9d0cc05fd42a51418c501c3e800"} Jul 02 17:38:23.142063 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.142058 2556 scope.go:117] "RemoveContainer" containerID="36c1265b73ac2e7b34fd34b3a4ab48fabe01ba06d7517313b726030920477456" Jul 02 17:38:23.150822 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.150807 2556 scope.go:117] "RemoveContainer" containerID="e0cc4ebe1f76a16a6ac63f2290f7f1a82a3e7fea5e063c7730a4dbd17317ac7c" Jul 02 17:38:23.160436 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.160421 2556 scope.go:117] "RemoveContainer" containerID="36c1265b73ac2e7b34fd34b3a4ab48fabe01ba06d7517313b726030920477456" Jul 02 17:38:23.160714 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:38:23.160694 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36c1265b73ac2e7b34fd34b3a4ab48fabe01ba06d7517313b726030920477456\": container with ID starting with 36c1265b73ac2e7b34fd34b3a4ab48fabe01ba06d7517313b726030920477456 not found: ID does not exist" containerID="36c1265b73ac2e7b34fd34b3a4ab48fabe01ba06d7517313b726030920477456" Jul 02 17:38:23.160771 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.160724 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36c1265b73ac2e7b34fd34b3a4ab48fabe01ba06d7517313b726030920477456"} err="failed to get container status \"36c1265b73ac2e7b34fd34b3a4ab48fabe01ba06d7517313b726030920477456\": rpc error: code = NotFound desc = could not find container \"36c1265b73ac2e7b34fd34b3a4ab48fabe01ba06d7517313b726030920477456\": container with ID starting with 36c1265b73ac2e7b34fd34b3a4ab48fabe01ba06d7517313b726030920477456 not found: ID does not exist" Jul 02 17:38:23.160771 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.160741 2556 scope.go:117] "RemoveContainer" containerID="e0cc4ebe1f76a16a6ac63f2290f7f1a82a3e7fea5e063c7730a4dbd17317ac7c" Jul 02 17:38:23.161023 ip-10-0-129-102 kubenswrapper[2556]: E0702 17:38:23.161004 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0cc4ebe1f76a16a6ac63f2290f7f1a82a3e7fea5e063c7730a4dbd17317ac7c\": container with ID starting with e0cc4ebe1f76a16a6ac63f2290f7f1a82a3e7fea5e063c7730a4dbd17317ac7c not found: ID does not exist" containerID="e0cc4ebe1f76a16a6ac63f2290f7f1a82a3e7fea5e063c7730a4dbd17317ac7c" Jul 02 17:38:23.161089 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.161032 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0cc4ebe1f76a16a6ac63f2290f7f1a82a3e7fea5e063c7730a4dbd17317ac7c"} err="failed to get container status \"e0cc4ebe1f76a16a6ac63f2290f7f1a82a3e7fea5e063c7730a4dbd17317ac7c\": rpc error: code = NotFound desc = could not find container \"e0cc4ebe1f76a16a6ac63f2290f7f1a82a3e7fea5e063c7730a4dbd17317ac7c\": container with ID starting with e0cc4ebe1f76a16a6ac63f2290f7f1a82a3e7fea5e063c7730a4dbd17317ac7c not found: ID does not exist" Jul 02 17:38:23.230330 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.230306 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqf4d\" (UniqueName: \"kubernetes.io/projected/a8856a07-7f1e-4359-845f-e041ca002854-kube-api-access-kqf4d\") pod \"a8856a07-7f1e-4359-845f-e041ca002854\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " Jul 02 17:38:23.230459 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.230343 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8856a07-7f1e-4359-845f-e041ca002854-istio-token\") pod \"a8856a07-7f1e-4359-845f-e041ca002854\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " Jul 02 17:38:23.230459 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.230381 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8856a07-7f1e-4359-845f-e041ca002854-istio-podinfo\") pod \"a8856a07-7f1e-4359-845f-e041ca002854\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " Jul 02 17:38:23.230585 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.230488 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-credential-socket\") pod \"a8856a07-7f1e-4359-845f-e041ca002854\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " Jul 02 17:38:23.230585 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.230545 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-istio-data\") pod \"a8856a07-7f1e-4359-845f-e041ca002854\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " Jul 02 17:38:23.230691 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.230660 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-workload-certs\") pod \"a8856a07-7f1e-4359-845f-e041ca002854\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " Jul 02 17:38:23.230745 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.230690 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-workload-socket\") pod \"a8856a07-7f1e-4359-845f-e041ca002854\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " Jul 02 17:38:23.230745 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.230715 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8856a07-7f1e-4359-845f-e041ca002854-istiod-ca-cert\") pod \"a8856a07-7f1e-4359-845f-e041ca002854\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " Jul 02 17:38:23.230840 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.230758 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-istio-envoy\") pod \"a8856a07-7f1e-4359-845f-e041ca002854\" (UID: \"a8856a07-7f1e-4359-845f-e041ca002854\") " Jul 02 17:38:23.230840 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.230797 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a8856a07-7f1e-4359-845f-e041ca002854" (UID: "a8856a07-7f1e-4359-845f-e041ca002854"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:38:23.230984 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.230939 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a8856a07-7f1e-4359-845f-e041ca002854" (UID: "a8856a07-7f1e-4359-845f-e041ca002854"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:38:23.231043 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.231017 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a8856a07-7f1e-4359-845f-e041ca002854" (UID: "a8856a07-7f1e-4359-845f-e041ca002854"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:38:23.231131 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.231111 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-workload-certs\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:23.231222 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.231135 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-workload-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:23.231222 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.231150 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-credential-socket\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:23.231222 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.231130 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8856a07-7f1e-4359-845f-e041ca002854-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a8856a07-7f1e-4359-845f-e041ca002854" (UID: "a8856a07-7f1e-4359-845f-e041ca002854"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:38:23.231429 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.231408 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-istio-data" (OuterVolumeSpecName: "istio-data") pod "a8856a07-7f1e-4359-845f-e041ca002854" (UID: "a8856a07-7f1e-4359-845f-e041ca002854"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:38:23.232913 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.232890 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8856a07-7f1e-4359-845f-e041ca002854-kube-api-access-kqf4d" (OuterVolumeSpecName: "kube-api-access-kqf4d") pod "a8856a07-7f1e-4359-845f-e041ca002854" (UID: "a8856a07-7f1e-4359-845f-e041ca002854"). InnerVolumeSpecName "kube-api-access-kqf4d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:38:23.233032 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.232888 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8856a07-7f1e-4359-845f-e041ca002854-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a8856a07-7f1e-4359-845f-e041ca002854" (UID: "a8856a07-7f1e-4359-845f-e041ca002854"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 17:38:23.233032 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.233002 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8856a07-7f1e-4359-845f-e041ca002854-istio-token" (OuterVolumeSpecName: "istio-token") pod "a8856a07-7f1e-4359-845f-e041ca002854" (UID: "a8856a07-7f1e-4359-845f-e041ca002854"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:38:23.233151 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.233040 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a8856a07-7f1e-4359-845f-e041ca002854" (UID: "a8856a07-7f1e-4359-845f-e041ca002854"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:38:23.332123 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.332095 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8856a07-7f1e-4359-845f-e041ca002854-istiod-ca-cert\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:23.332123 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.332117 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-istio-envoy\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:23.332123 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.332130 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqf4d\" (UniqueName: \"kubernetes.io/projected/a8856a07-7f1e-4359-845f-e041ca002854-kube-api-access-kqf4d\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:23.332410 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.332138 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8856a07-7f1e-4359-845f-e041ca002854-istio-token\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:23.332410 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.332147 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8856a07-7f1e-4359-845f-e041ca002854-istio-podinfo\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:23.332410 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.332154 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8856a07-7f1e-4359-845f-e041ca002854-istio-data\") on node \"ip-10-0-129-102.ec2.internal\" DevicePath \"\"" Jul 02 17:38:23.426561 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.426468 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kfbp2_2f06eafc-832e-40c1-9117-de241dbf4209/manager/0.log" Jul 02 17:38:23.463702 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.463675 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk"] Jul 02 17:38:23.469440 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.469413 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-4421aa-openshift-default-f7bc45cdc-h6vqk"] Jul 02 17:38:23.574076 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:23.574043 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-d6gv7_36675e91-c4ed-46d0-ab14-43cf33c2216b/manager/0.log" Jul 02 17:38:24.217883 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:24.217856 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-m984c_b4f673c6-32fb-4165-be0f-d398f6aa2964/server/0.log" Jul 02 17:38:24.355181 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:24.355139 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-zwblr_b660efa9-505d-4c85-9dba-7f86e21ea744/manager/0.log" Jul 02 17:38:24.631789 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:24.631762 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845f459c56-6skck_6e3d5582-387e-4bbd-a694-a63b3b65bf2e/copy-manifests/0.log" Jul 02 17:38:24.639287 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:24.639265 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845f459c56-6skck_6e3d5582-387e-4bbd-a694-a63b3b65bf2e/manager/0.log" Jul 02 17:38:25.189892 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:25.189863 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8856a07-7f1e-4359-845f-e041ca002854" path="/var/lib/kubelet/pods/a8856a07-7f1e-4359-845f-e041ca002854/volumes" Jul 02 17:38:26.435256 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:26.435230 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-n7b24_b650dd65-4cfa-48ac-adca-2cac5862b80c/manager/0.log" Jul 02 17:38:26.575646 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:26.575616 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-lqrmq_361c3f42-9644-402f-bd99-97d08fe0ff53/manager/0.log" Jul 02 17:38:26.816424 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:26.816387 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-r68bn_53532975-54fb-4f3e-bd1a-c6fff83395e3/registry-server/0.log" Jul 02 17:38:26.948933 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:26.948902 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-x9j96_0011b5c4-160c-4e50-aa00-9611104702e2/manager/0.log" Jul 02 17:38:27.090577 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:27.090491 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-gn5fl_5d7f90bf-6317-42be-8e5b-a81e2a5f981c/limitador/0.log" Jul 02 17:38:27.226251 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:27.226222 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-42dl9_dd47754e-cfcc-4858-abb4-de54dc86fdef/manager/0.log" Jul 02 17:38:27.588398 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:27.588363 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-656c84dccdqgg22_7bad9014-0315-437b-a0d6-7504cd44aca8/istio-proxy/0.log" Jul 02 17:38:27.936898 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:27.936873 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bc8ccd95b-tt5nf_60a61adf-bf87-41bb-b5e2-631658ab2267/kube-auth-proxy/0.log" Jul 02 17:38:28.056946 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:28.056911 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-57xrf_b6335ba4-07ee-4581-9d37-5d3755535085/istio-proxy/1.log" Jul 02 17:38:29.221516 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:29.221484 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-dcq4b_6b4077ad-b97c-47a3-8d31-f8cb82e9a88a/main/0.log" Jul 02 17:38:33.950210 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.950179 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tt4vl/must-gather-gplkn"] Jul 02 17:38:33.950591 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.950582 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d4f453-f5fa-4860-beb9-2f0820144eec" containerName="istio-proxy" Jul 02 17:38:33.950636 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.950594 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d4f453-f5fa-4860-beb9-2f0820144eec" containerName="istio-proxy" Jul 02 17:38:33.950636 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.950611 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d4f453-f5fa-4860-beb9-2f0820144eec" containerName="istio-proxy" Jul 02 17:38:33.950636 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.950616 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d4f453-f5fa-4860-beb9-2f0820144eec" containerName="istio-proxy" Jul 02 17:38:33.950636 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.950625 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8856a07-7f1e-4359-845f-e041ca002854" containerName="istio-proxy" Jul 02 17:38:33.950636 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.950630 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8856a07-7f1e-4359-845f-e041ca002854" containerName="istio-proxy" Jul 02 17:38:33.950797 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.950640 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8856a07-7f1e-4359-845f-e041ca002854" containerName="istio-proxy" Jul 02 17:38:33.950797 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.950646 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8856a07-7f1e-4359-845f-e041ca002854" containerName="istio-proxy" Jul 02 17:38:33.950797 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.950721 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8856a07-7f1e-4359-845f-e041ca002854" containerName="istio-proxy" Jul 02 17:38:33.950797 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.950729 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="08d4f453-f5fa-4860-beb9-2f0820144eec" containerName="istio-proxy" Jul 02 17:38:33.950797 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.950736 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8856a07-7f1e-4359-845f-e041ca002854" containerName="istio-proxy" Jul 02 17:38:33.950797 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.950743 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f7702e4-91b1-4dc6-82a0-6901209c18e4" containerName="istio-proxy" Jul 02 17:38:33.950998 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.950863 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="08d4f453-f5fa-4860-beb9-2f0820144eec" containerName="istio-proxy" Jul 02 17:38:33.954203 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.954185 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tt4vl/must-gather-gplkn" Jul 02 17:38:33.956741 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.956720 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tt4vl\"/\"default-dockercfg-w8jbn\"" Jul 02 17:38:33.957705 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.957690 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tt4vl\"/\"openshift-service-ca.crt\"" Jul 02 17:38:33.957753 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.957728 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tt4vl\"/\"kube-root-ca.crt\"" Jul 02 17:38:33.962825 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:33.962805 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tt4vl/must-gather-gplkn"] Jul 02 17:38:34.030584 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:34.030552 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8024-0f0d-40e5-a373-a48e16edf108-must-gather-output\") pod \"must-gather-gplkn\" (UID: \"5ecb8024-0f0d-40e5-a373-a48e16edf108\") " pod="openshift-must-gather-tt4vl/must-gather-gplkn" Jul 02 17:38:34.030727 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:34.030651 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6ffv\" (UniqueName: \"kubernetes.io/projected/5ecb8024-0f0d-40e5-a373-a48e16edf108-kube-api-access-t6ffv\") pod \"must-gather-gplkn\" (UID: \"5ecb8024-0f0d-40e5-a373-a48e16edf108\") " pod="openshift-must-gather-tt4vl/must-gather-gplkn" Jul 02 17:38:34.131541 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:34.131505 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8024-0f0d-40e5-a373-a48e16edf108-must-gather-output\") pod \"must-gather-gplkn\" (UID: \"5ecb8024-0f0d-40e5-a373-a48e16edf108\") " pod="openshift-must-gather-tt4vl/must-gather-gplkn" Jul 02 17:38:34.131718 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:34.131579 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6ffv\" (UniqueName: \"kubernetes.io/projected/5ecb8024-0f0d-40e5-a373-a48e16edf108-kube-api-access-t6ffv\") pod \"must-gather-gplkn\" (UID: \"5ecb8024-0f0d-40e5-a373-a48e16edf108\") " pod="openshift-must-gather-tt4vl/must-gather-gplkn" Jul 02 17:38:34.131914 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:34.131893 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5ecb8024-0f0d-40e5-a373-a48e16edf108-must-gather-output\") pod \"must-gather-gplkn\" (UID: \"5ecb8024-0f0d-40e5-a373-a48e16edf108\") " pod="openshift-must-gather-tt4vl/must-gather-gplkn" Jul 02 17:38:34.139581 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:34.139552 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6ffv\" (UniqueName: \"kubernetes.io/projected/5ecb8024-0f0d-40e5-a373-a48e16edf108-kube-api-access-t6ffv\") pod \"must-gather-gplkn\" (UID: \"5ecb8024-0f0d-40e5-a373-a48e16edf108\") " pod="openshift-must-gather-tt4vl/must-gather-gplkn" Jul 02 17:38:34.264713 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:34.264689 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tt4vl/must-gather-gplkn" Jul 02 17:38:34.398326 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:34.398297 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tt4vl/must-gather-gplkn"] Jul 02 17:38:34.400436 ip-10-0-129-102 kubenswrapper[2556]: W0702 17:38:34.400411 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ecb8024_0f0d_40e5_a373_a48e16edf108.slice/crio-6c40a0d33b42ee1812c9567ed31a66dcd2213bd1f465324ab86216cc7e219035 WatchSource:0}: Error finding container 6c40a0d33b42ee1812c9567ed31a66dcd2213bd1f465324ab86216cc7e219035: Status 404 returned error can't find the container with id 6c40a0d33b42ee1812c9567ed31a66dcd2213bd1f465324ab86216cc7e219035 Jul 02 17:38:34.402424 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:34.402408 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 17:38:35.206996 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:35.206959 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tt4vl/must-gather-gplkn" event={"ID":"5ecb8024-0f0d-40e5-a373-a48e16edf108","Type":"ContainerStarted","Data":"6c40a0d33b42ee1812c9567ed31a66dcd2213bd1f465324ab86216cc7e219035"} Jul 02 17:38:39.230733 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:39.230692 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tt4vl/must-gather-gplkn" event={"ID":"5ecb8024-0f0d-40e5-a373-a48e16edf108","Type":"ContainerStarted","Data":"a385d354caf7f8f44538e4aae950eccbef9ceffb67684f746cb69ecdba6ee4f4"} Jul 02 17:38:39.231135 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:39.230746 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tt4vl/must-gather-gplkn" event={"ID":"5ecb8024-0f0d-40e5-a373-a48e16edf108","Type":"ContainerStarted","Data":"5249aedb89cee51c271443d6c1acb7751fe2b5127d294768213cbc750d2aa361"} Jul 02 17:38:39.248623 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:39.248561 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tt4vl/must-gather-gplkn" podStartSLOduration=2.159130701 podStartE2EDuration="6.24853765s" totalImagesPullingTime="4.089406949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:38:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:38:39.245732614 +0000 UTC m=+2652.558606261" watchObservedRunningTime="2026-07-02 17:38:39.24853765 +0000 UTC m=+2652.561411296" Jul 02 17:38:40.027725 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:40.027691 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-76pzw_3a271183-1ef5-49c2-a88f-7ea46b2b8864/global-pull-secret-syncer/0.log" Jul 02 17:38:40.253263 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:40.253226 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wbgfs_53d27a00-961d-4ea1-9312-39cfc71e8bb5/konnectivity-agent/0.log" Jul 02 17:38:40.273723 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:40.273688 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-102.ec2.internal_b66c94e3b0824384de48db496e306579/haproxy/0.log" Jul 02 17:38:45.863285 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:45.863253 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-n7b24_b650dd65-4cfa-48ac-adca-2cac5862b80c/manager/0.log" Jul 02 17:38:45.895089 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:45.895054 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-lqrmq_361c3f42-9644-402f-bd99-97d08fe0ff53/manager/0.log" Jul 02 17:38:45.973292 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:45.973257 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-r68bn_53532975-54fb-4f3e-bd1a-c6fff83395e3/registry-server/0.log" Jul 02 17:38:46.230227 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:46.228052 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-x9j96_0011b5c4-160c-4e50-aa00-9611104702e2/manager/0.log" Jul 02 17:38:46.256106 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:46.256080 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-gn5fl_5d7f90bf-6317-42be-8e5b-a81e2a5f981c/limitador/0.log" Jul 02 17:38:46.380182 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:46.379574 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-42dl9_dd47754e-cfcc-4858-abb4-de54dc86fdef/manager/0.log" Jul 02 17:38:48.594830 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:48.594802 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-smszq_4582a064-0ea5-4c77-9e29-ed41109a0325/node-exporter/0.log" Jul 02 17:38:48.615304 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:48.615271 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-smszq_4582a064-0ea5-4c77-9e29-ed41109a0325/kube-rbac-proxy/0.log" Jul 02 17:38:48.635375 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:48.635340 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-smszq_4582a064-0ea5-4c77-9e29-ed41109a0325/init-textfile/0.log" Jul 02 17:38:48.964549 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:48.964468 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7ddc8784d4-t8hhj_bd1e6dfc-555f-4f6e-95d2-bae33e28e7c1/prometheus-operator-admission-webhook/0.log" Jul 02 17:38:49.072937 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:49.072906 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d45fb65c-wtv7z_ad94a6a3-4de7-472c-b756-fb09c79dcbd5/thanos-query/0.log" Jul 02 17:38:49.094573 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:49.094541 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d45fb65c-wtv7z_ad94a6a3-4de7-472c-b756-fb09c79dcbd5/kube-rbac-proxy-web/0.log" Jul 02 17:38:49.113090 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:49.113062 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d45fb65c-wtv7z_ad94a6a3-4de7-472c-b756-fb09c79dcbd5/kube-rbac-proxy/0.log" Jul 02 17:38:49.132930 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:49.132902 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d45fb65c-wtv7z_ad94a6a3-4de7-472c-b756-fb09c79dcbd5/prom-label-proxy/0.log" Jul 02 17:38:49.154674 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:49.154643 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d45fb65c-wtv7z_ad94a6a3-4de7-472c-b756-fb09c79dcbd5/kube-rbac-proxy-rules/0.log" Jul 02 17:38:49.174201 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:49.174154 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d45fb65c-wtv7z_ad94a6a3-4de7-472c-b756-fb09c79dcbd5/kube-rbac-proxy-metrics/0.log" Jul 02 17:38:50.005792 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.005742 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp"] Jul 02 17:38:50.012762 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.012735 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.023186 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.023143 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp"] Jul 02 17:38:50.103289 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.103258 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bb14be7b-c51a-44da-b3ea-1098c35a1057-proc\") pod \"perf-node-gather-daemonset-gt2qp\" (UID: \"bb14be7b-c51a-44da-b3ea-1098c35a1057\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.103484 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.103304 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb14be7b-c51a-44da-b3ea-1098c35a1057-lib-modules\") pod \"perf-node-gather-daemonset-gt2qp\" (UID: \"bb14be7b-c51a-44da-b3ea-1098c35a1057\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.103484 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.103336 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bb14be7b-c51a-44da-b3ea-1098c35a1057-podres\") pod \"perf-node-gather-daemonset-gt2qp\" (UID: \"bb14be7b-c51a-44da-b3ea-1098c35a1057\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.103484 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.103459 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb14be7b-c51a-44da-b3ea-1098c35a1057-sys\") pod \"perf-node-gather-daemonset-gt2qp\" (UID: \"bb14be7b-c51a-44da-b3ea-1098c35a1057\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.103636 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.103512 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf2kn\" (UniqueName: \"kubernetes.io/projected/bb14be7b-c51a-44da-b3ea-1098c35a1057-kube-api-access-lf2kn\") pod \"perf-node-gather-daemonset-gt2qp\" (UID: \"bb14be7b-c51a-44da-b3ea-1098c35a1057\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.204873 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.204828 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bb14be7b-c51a-44da-b3ea-1098c35a1057-proc\") pod \"perf-node-gather-daemonset-gt2qp\" (UID: \"bb14be7b-c51a-44da-b3ea-1098c35a1057\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.205074 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.204910 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb14be7b-c51a-44da-b3ea-1098c35a1057-lib-modules\") pod \"perf-node-gather-daemonset-gt2qp\" (UID: \"bb14be7b-c51a-44da-b3ea-1098c35a1057\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.205074 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.204947 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bb14be7b-c51a-44da-b3ea-1098c35a1057-podres\") pod \"perf-node-gather-daemonset-gt2qp\" (UID: \"bb14be7b-c51a-44da-b3ea-1098c35a1057\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.205074 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.205032 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb14be7b-c51a-44da-b3ea-1098c35a1057-sys\") pod \"perf-node-gather-daemonset-gt2qp\" (UID: \"bb14be7b-c51a-44da-b3ea-1098c35a1057\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.205074 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.205064 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf2kn\" (UniqueName: \"kubernetes.io/projected/bb14be7b-c51a-44da-b3ea-1098c35a1057-kube-api-access-lf2kn\") pod \"perf-node-gather-daemonset-gt2qp\" (UID: \"bb14be7b-c51a-44da-b3ea-1098c35a1057\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.205591 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.205559 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bb14be7b-c51a-44da-b3ea-1098c35a1057-proc\") pod \"perf-node-gather-daemonset-gt2qp\" (UID: \"bb14be7b-c51a-44da-b3ea-1098c35a1057\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.205749 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.205727 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb14be7b-c51a-44da-b3ea-1098c35a1057-lib-modules\") pod \"perf-node-gather-daemonset-gt2qp\" (UID: \"bb14be7b-c51a-44da-b3ea-1098c35a1057\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.205842 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.205823 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bb14be7b-c51a-44da-b3ea-1098c35a1057-podres\") pod \"perf-node-gather-daemonset-gt2qp\" (UID: \"bb14be7b-c51a-44da-b3ea-1098c35a1057\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.205917 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.205881 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb14be7b-c51a-44da-b3ea-1098c35a1057-sys\") pod \"perf-node-gather-daemonset-gt2qp\" (UID: \"bb14be7b-c51a-44da-b3ea-1098c35a1057\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.214275 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.214241 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf2kn\" (UniqueName: \"kubernetes.io/projected/bb14be7b-c51a-44da-b3ea-1098c35a1057-kube-api-access-lf2kn\") pod \"perf-node-gather-daemonset-gt2qp\" (UID: \"bb14be7b-c51a-44da-b3ea-1098c35a1057\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.336736 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.336660 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:50.529774 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.525998 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp"] Jul 02 17:38:50.939395 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.939313 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/2.log" Jul 02 17:38:50.947408 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:50.947374 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-njvpd_81ef2a97-c51e-4d07-b26d-083e4e348612/console-operator/3.log" Jul 02 17:38:51.320688 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:51.320648 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" event={"ID":"bb14be7b-c51a-44da-b3ea-1098c35a1057","Type":"ContainerStarted","Data":"e0b9b54fb4384ddc38b0f333a304ae336a977a6bc457899e78dc1aadc19c8dbf"} Jul 02 17:38:51.321268 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:51.321241 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" event={"ID":"bb14be7b-c51a-44da-b3ea-1098c35a1057","Type":"ContainerStarted","Data":"32941ae7aa6fd0774cf14be3bf9590dbba37b779c25ba083e7a80eb6d29a44f7"} Jul 02 17:38:51.321461 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:51.321446 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:51.336578 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:51.336528 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" podStartSLOduration=2.336508901 podStartE2EDuration="2.336508901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 17:38:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:38:51.335439075 +0000 UTC m=+2664.648312724" watchObservedRunningTime="2026-07-02 17:38:51.336508901 +0000 UTC m=+2664.649382546" Jul 02 17:38:51.456224 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:51.456191 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5499b6fb8f-fwk25_0ee604fa-e2f0-4616-978a-ef689e30fd4a/download-server/0.log" Jul 02 17:38:51.991917 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:51.991879 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-69d9f679db-g2l5k_edc0b924-909a-4f9a-87d6-1c3d0426cd3f/volume-data-source-validator/0.log" Jul 02 17:38:53.000047 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:53.000014 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-76lqz_eca22f13-1929-48b6-90fd-5d82f58f6017/dns/0.log" Jul 02 17:38:53.019833 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:53.019806 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-76lqz_eca22f13-1929-48b6-90fd-5d82f58f6017/kube-rbac-proxy/0.log" Jul 02 17:38:53.186060 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:53.186028 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6c5gj_28783fce-657d-4461-84ca-ae848eb09b15/dns-node-resolver/0.log" Jul 02 17:38:53.815582 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:53.815546 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-c9f9f68b6-xmx27_90c0aef6-7f79-4c5e-96e5-811086bcad5e/registry/0.log" Jul 02 17:38:53.836939 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:53.836913 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-clww4_dfb32fb2-9a93-4405-be31-8c26f66032e1/node-ca/0.log" Jul 02 17:38:54.806029 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:54.805997 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-656c84dccdqgg22_7bad9014-0315-437b-a0d6-7504cd44aca8/istio-proxy/0.log" Jul 02 17:38:55.042910 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:55.042871 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bc8ccd95b-tt5nf_60a61adf-bf87-41bb-b5e2-631658ab2267/kube-auth-proxy/0.log" Jul 02 17:38:55.070339 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:55.070261 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-57xrf_b6335ba4-07ee-4581-9d37-5d3755535085/istio-proxy/0.log" Jul 02 17:38:55.131968 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:55.131942 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-57xrf_b6335ba4-07ee-4581-9d37-5d3755535085/istio-proxy/1.log" Jul 02 17:38:55.936143 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:55.936114 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9qqxh_6cec11ca-b0a2-4bd4-ae8e-bb2e6ac12df0/serve-healthcheck-canary/0.log" Jul 02 17:38:56.447545 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:56.447507 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5f67b55555-gzntn_4d1a424d-f615-4619-9406-42979581d953/insights-operator/0.log" Jul 02 17:38:56.448645 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:56.448609 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5f67b55555-gzntn_4d1a424d-f615-4619-9406-42979581d953/insights-operator/1.log" Jul 02 17:38:56.471399 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:56.471363 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4l2sg_82bb767c-600e-4125-bf49-254356619b37/kube-rbac-proxy/0.log" Jul 02 17:38:56.492834 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:56.492799 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4l2sg_82bb767c-600e-4125-bf49-254356619b37/exporter/0.log" Jul 02 17:38:56.513420 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:56.513393 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4l2sg_82bb767c-600e-4125-bf49-254356619b37/extractor/0.log" Jul 02 17:38:58.347870 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:58.347839 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-gt2qp" Jul 02 17:38:58.794447 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:58.794419 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kfbp2_2f06eafc-832e-40c1-9117-de241dbf4209/manager/0.log" Jul 02 17:38:59.032421 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:59.032391 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-d6gv7_36675e91-c4ed-46d0-ab14-43cf33c2216b/manager/0.log" Jul 02 17:38:59.311961 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:59.311935 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-m984c_b4f673c6-32fb-4165-be0f-d398f6aa2964/server/0.log" Jul 02 17:38:59.426598 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:59.426042 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-zwblr_b660efa9-505d-4c85-9dba-7f86e21ea744/manager/0.log" Jul 02 17:38:59.576432 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:59.576360 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845f459c56-6skck_6e3d5582-387e-4bbd-a694-a63b3b65bf2e/manager/0.log" Jul 02 17:38:59.598192 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:38:59.598149 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845f459c56-6skck_6e3d5582-387e-4bbd-a694-a63b3b65bf2e/copy-manifests/0.log" Jul 02 17:39:00.298178 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:00.298128 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-656c84dccdqgg22_7bad9014-0315-437b-a0d6-7504cd44aca8/istio-proxy/0.log" Jul 02 17:39:00.519590 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:00.519563 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bc8ccd95b-tt5nf_60a61adf-bf87-41bb-b5e2-631658ab2267/kube-auth-proxy/0.log" Jul 02 17:39:00.543347 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:00.543312 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-57xrf_b6335ba4-07ee-4581-9d37-5d3755535085/istio-proxy/0.log" Jul 02 17:39:00.593284 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:00.593248 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-57xrf_b6335ba4-07ee-4581-9d37-5d3755535085/istio-proxy/1.log" Jul 02 17:39:01.920265 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:01.920224 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-656c84dccdqgg22_7bad9014-0315-437b-a0d6-7504cd44aca8/istio-proxy/0.log" Jul 02 17:39:02.135381 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:02.135339 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bc8ccd95b-tt5nf_60a61adf-bf87-41bb-b5e2-631658ab2267/kube-auth-proxy/0.log" Jul 02 17:39:02.154372 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:02.154338 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-57xrf_b6335ba4-07ee-4581-9d37-5d3755535085/istio-proxy/0.log" Jul 02 17:39:02.218182 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:02.218090 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-57xrf_b6335ba4-07ee-4581-9d37-5d3755535085/istio-proxy/1.log" Jul 02 17:39:05.988357 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:05.988303 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-rq4sf_570da710-50a2-4118-bb22-ea95e478156f/migrator/0.log" Jul 02 17:39:06.007998 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:06.007975 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-rq4sf_570da710-50a2-4118-bb22-ea95e478156f/graceful-termination/0.log" Jul 02 17:39:07.325373 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:07.325340 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4mb2b_cce0691f-7213-4afa-b73e-d22536b1861a/kube-multus/0.log" Jul 02 17:39:07.502708 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:07.502685 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fnsfd_aa88be7d-b349-4071-8f78-2cdafc2fd875/kube-multus-additional-cni-plugins/0.log" Jul 02 17:39:07.522955 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:07.522926 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fnsfd_aa88be7d-b349-4071-8f78-2cdafc2fd875/egress-router-binary-copy/0.log" Jul 02 17:39:07.543893 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:07.543868 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fnsfd_aa88be7d-b349-4071-8f78-2cdafc2fd875/cni-plugins/0.log" Jul 02 17:39:07.582413 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:07.582353 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fnsfd_aa88be7d-b349-4071-8f78-2cdafc2fd875/bond-cni-plugin/0.log" Jul 02 17:39:07.602888 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:07.602862 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fnsfd_aa88be7d-b349-4071-8f78-2cdafc2fd875/routeoverride-cni/0.log" Jul 02 17:39:07.626596 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:07.626576 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fnsfd_aa88be7d-b349-4071-8f78-2cdafc2fd875/whereabouts-cni-bincopy/0.log" Jul 02 17:39:07.654299 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:07.654283 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fnsfd_aa88be7d-b349-4071-8f78-2cdafc2fd875/whereabouts-cni/0.log" Jul 02 17:39:07.918746 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:07.918663 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bzrql_6c35723b-8e1e-4682-b963-70e3266a6a71/network-metrics-daemon/0.log" Jul 02 17:39:07.956275 ip-10-0-129-102 kubenswrapper[2556]: I0702 17:39:07.956247 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bzrql_6c35723b-8e1e-4682-b963-70e3266a6a71/kube-rbac-proxy/0.log"