Jun 08 15:30:47.389625 ip-10-0-134-81 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 08 15:30:47.389823 ip-10-0-134-81 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 08 15:30:47.389839 ip-10-0-134-81 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 08 15:30:47.390185 ip-10-0-134-81 systemd[1]: Failed to start Kubernetes Kubelet. Jun 08 15:30:58.551722 ip-10-0-134-81 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 08 15:30:58.551736 ip-10-0-134-81 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 5614ccbd5b40447eb4187b998096662d -- Jun 08 15:33:29.799800 ip-10-0-134-81 systemd[1]: Starting Kubernetes Kubelet... Jun 08 15:33:30.182604 ip-10-0-134-81 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 15:33:30.182604 ip-10-0-134-81 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 08 15:33:30.182604 ip-10-0-134-81 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 15:33:30.182604 ip-10-0-134-81 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 08 15:33:30.182604 ip-10-0-134-81 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 15:33:30.183554 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.183454 2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 08 15:33:30.191042 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191022 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 15:33:30.191042 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191038 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 15:33:30.191042 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191042 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 15:33:30.191042 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191046 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 15:33:30.191042 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191048 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191052 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191055 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191057 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191061 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191064 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191066 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191069 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191071 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191074 2557 feature_gate.go:328] unrecognized feature gate: Example Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191077 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191080 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191084 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191088 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191091 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191094 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191097 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191100 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191102 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 15:33:30.191242 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191105 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191107 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191110 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191113 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191116 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191119 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191122 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191124 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191127 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191130 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191132 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191135 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191137 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191140 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191142 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191146 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191149 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191152 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191154 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191157 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 15:33:30.191725 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191160 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191162 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191165 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191168 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191170 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191173 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191175 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191178 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191181 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191183 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191185 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191188 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191191 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191194 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191197 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191199 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191202 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191204 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191207 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191210 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 15:33:30.192254 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191212 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191215 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191217 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191220 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191222 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191225 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191227 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191231 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191234 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191237 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191239 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191242 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191244 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191247 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191250 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191254 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191256 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191260 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191264 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 15:33:30.192757 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191268 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191272 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191276 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191279 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191282 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191285 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191287 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191290 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191293 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191296 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191298 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191301 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191304 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191306 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191309 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191311 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.191314 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192748 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192756 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 15:33:30.193224 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192761 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192765 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192768 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192771 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192774 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192776 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192779 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192782 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192786 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192789 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192792 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192796 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192798 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192801 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192804 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192808 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192810 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192813 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192816 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 15:33:30.193712 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192818 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192821 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192824 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192826 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192829 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192831 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192834 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192836 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192839 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192841 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192844 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192846 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192848 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192851 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192854 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192857 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192861 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192864 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192867 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 15:33:30.194179 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192871 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192873 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192876 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192879 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192881 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192884 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192887 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192890 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192893 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192896 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192900 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192902 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192905 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192908 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192910 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192913 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192915 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192917 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192920 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192923 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 15:33:30.194655 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192925 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192928 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192930 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192933 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192935 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192938 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192940 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192943 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192946 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192949 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192951 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192954 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192957 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192959 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192962 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192964 2557 feature_gate.go:328] unrecognized feature gate: Example Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192967 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192970 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192973 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192975 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192978 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 15:33:30.195144 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192981 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192983 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192986 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192989 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192992 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192994 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.192997 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193000 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193002 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193004 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193007 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193010 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193012 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193014 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193017 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193019 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193022 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193024 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193378 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193383 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 15:33:30.195676 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193386 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193389 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193391 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193394 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193397 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193399 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193402 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193405 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193407 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193410 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193412 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193415 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193427 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193431 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193434 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193438 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193440 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193444 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193446 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193449 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 15:33:30.196165 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193451 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193454 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193456 2557 feature_gate.go:328] unrecognized feature gate: Example Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193472 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193475 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193478 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193481 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193483 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193486 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193489 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193493 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193497 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193500 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193503 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193506 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193509 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193511 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193514 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193516 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193519 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 15:33:30.196668 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193521 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193524 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193526 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193529 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193533 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193535 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193538 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193542 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193544 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193547 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193549 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193552 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193554 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193556 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193559 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193562 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193564 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193567 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193569 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 15:33:30.197193 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193571 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193574 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193576 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193579 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193581 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193584 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193589 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193592 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193595 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193598 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193601 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193603 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193606 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193608 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193610 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193613 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193615 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193618 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193621 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 15:33:30.197694 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193624 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193626 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193630 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193632 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193635 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193637 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193640 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193642 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193645 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193649 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193652 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193654 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193658 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193661 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193663 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193666 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193668 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193671 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.193674 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193745 2557 flags.go:64] FLAG: --address="0.0.0.0" Jun 08 15:33:30.198161 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193752 2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193761 2557 flags.go:64] FLAG: --anonymous-auth="true" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193775 2557 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193782 2557 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193786 2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193790 2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193794 2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193797 2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193800 2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193803 2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193806 2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193810 2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193815 2557 flags.go:64] FLAG: --cgroup-root="" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193818 2557 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193821 2557 flags.go:64] FLAG: --client-ca-file="" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193825 2557 flags.go:64] FLAG: --cloud-provider="external" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193828 2557 flags.go:64] FLAG: --cluster-dns="[]" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193832 2557 flags.go:64] FLAG: --cluster-domain="" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193835 2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193838 2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193841 2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193845 2557 flags.go:64] FLAG: --container-log-max-files="5" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193848 2557 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 08 15:33:30.198649 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193851 2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193854 2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193857 2557 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193860 2557 flags.go:64] FLAG: --contention-profiling="false" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193863 2557 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193866 2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193869 2557 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193872 2557 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193876 2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193879 2557 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193881 2557 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193884 2557 flags.go:64] FLAG: --enable-load-reader="false" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193887 2557 flags.go:64] FLAG: --enable-server="true" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193890 2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193894 2557 flags.go:64] FLAG: --event-burst="100" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193897 2557 flags.go:64] FLAG: --event-qps="50" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193899 2557 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193902 2557 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193905 2557 flags.go:64] FLAG: --eviction-hard="" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193909 2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193912 2557 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193918 2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193921 2557 flags.go:64] FLAG: --eviction-soft="" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193924 2557 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193927 2557 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 08 15:33:30.199200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193930 2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193933 2557 flags.go:64] FLAG: --experimental-mounter-path="" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193936 2557 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193939 2557 flags.go:64] FLAG: --fail-swap-on="true" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193942 2557 flags.go:64] FLAG: --feature-gates="" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193945 2557 flags.go:64] FLAG: --file-check-frequency="20s" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193948 2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193952 2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193955 2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193958 2557 flags.go:64] FLAG: --healthz-port="10248" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193961 2557 flags.go:64] FLAG: --help="false" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193963 2557 flags.go:64] FLAG: --hostname-override="ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193966 2557 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193969 2557 flags.go:64] FLAG: --http-check-frequency="20s" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193972 2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193975 2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193979 2557 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193982 2557 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193984 2557 flags.go:64] FLAG: --image-service-endpoint="" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193987 2557 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193990 2557 flags.go:64] FLAG: --kube-api-burst="100" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193993 2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193996 2557 flags.go:64] FLAG: --kube-api-qps="50" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.193998 2557 flags.go:64] FLAG: --kube-reserved="" Jun 08 15:33:30.199867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194001 2557 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194004 2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194007 2557 flags.go:64] FLAG: --kubelet-cgroups="" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194010 2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194014 2557 flags.go:64] FLAG: --lock-file="" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194017 2557 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194020 2557 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194023 2557 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194028 2557 flags.go:64] FLAG: --log-json-split-stream="false" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194032 2557 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194035 2557 flags.go:64] FLAG: --log-text-split-stream="false" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194037 2557 flags.go:64] FLAG: --logging-format="text" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194040 2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194043 2557 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194046 2557 flags.go:64] FLAG: --manifest-url="" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194049 2557 flags.go:64] FLAG: --manifest-url-header="" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194053 2557 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194056 2557 flags.go:64] FLAG: --max-open-files="1000000" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194059 2557 flags.go:64] FLAG: --max-pods="110" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194062 2557 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194065 2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194068 2557 flags.go:64] FLAG: --memory-manager-policy="None" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194071 2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194074 2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194077 2557 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 08 15:33:30.200511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194080 2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194087 2557 flags.go:64] FLAG: --node-status-max-images="50" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194090 2557 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194093 2557 flags.go:64] FLAG: --oom-score-adj="-999" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194096 2557 flags.go:64] FLAG: --pod-cidr="" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194098 2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194104 2557 flags.go:64] FLAG: --pod-manifest-path="" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194107 2557 flags.go:64] FLAG: --pod-max-pids="-1" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194109 2557 flags.go:64] FLAG: --pods-per-core="0" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194112 2557 flags.go:64] FLAG: --port="10250" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194117 2557 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194121 2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d622befe3ae5699c" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194124 2557 flags.go:64] FLAG: --qos-reserved="" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194127 2557 flags.go:64] FLAG: --read-only-port="10255" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194130 2557 flags.go:64] FLAG: --register-node="true" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194133 2557 flags.go:64] FLAG: --register-with-taints="" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194137 2557 flags.go:64] FLAG: --registry-burst="10" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194140 2557 flags.go:64] FLAG: --registry-qps="5" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194143 2557 flags.go:64] FLAG: --reserved-cpus="" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194146 2557 flags.go:64] FLAG: --reserved-memory="" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194149 2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194152 2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194155 2557 flags.go:64] FLAG: --rotate-certificates="false" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194158 2557 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194161 2557 flags.go:64] FLAG: --runonce="false" Jun 08 15:33:30.201635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194163 2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194166 2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194169 2557 flags.go:64] FLAG: --seccomp-default="false" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194172 2557 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194175 2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194178 2557 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194181 2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194184 2557 flags.go:64] FLAG: --storage-driver-password="root" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194187 2557 flags.go:64] FLAG: --storage-driver-secure="false" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194190 2557 flags.go:64] FLAG: --storage-driver-table="stats" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194193 2557 flags.go:64] FLAG: --storage-driver-user="root" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194196 2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194199 2557 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194202 2557 flags.go:64] FLAG: --system-cgroups="" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194205 2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194210 2557 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194212 2557 flags.go:64] FLAG: --tls-cert-file="" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194216 2557 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194221 2557 flags.go:64] FLAG: --tls-min-version="" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194224 2557 flags.go:64] FLAG: --tls-private-key-file="" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194227 2557 flags.go:64] FLAG: --topology-manager-policy="none" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194230 2557 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194233 2557 flags.go:64] FLAG: --topology-manager-scope="container" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194236 2557 flags.go:64] FLAG: --v="2" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194240 2557 flags.go:64] FLAG: --version="false" Jun 08 15:33:30.202770 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194244 2557 flags.go:64] FLAG: --vmodule="" Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194247 2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.194250 2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194384 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194389 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194392 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194395 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194398 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194401 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194404 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194407 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194410 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194413 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194417 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194420 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194423 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194426 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194429 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194432 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 15:33:30.203891 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194435 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194438 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194440 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194443 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194446 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194450 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194453 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194456 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194473 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194477 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194480 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194483 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194486 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194489 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194493 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194496 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194499 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194501 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194504 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 15:33:30.204654 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194507 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194509 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194512 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194514 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194517 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194519 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194522 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194525 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194527 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194530 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194532 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194535 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194537 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194540 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194542 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194544 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194547 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194549 2557 feature_gate.go:328] unrecognized feature gate: Example Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194553 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194556 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 15:33:30.205195 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194559 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194562 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194565 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194567 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194569 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194572 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194575 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194578 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194581 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194583 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194586 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194589 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194591 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194594 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194597 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194599 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194602 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194604 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194606 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194609 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 15:33:30.205872 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194611 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194614 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194617 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194619 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194622 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194624 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194627 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194630 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194632 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194635 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194638 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194641 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194643 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194646 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194649 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194651 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194654 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194656 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194660 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194664 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 15:33:30.206630 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194667 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 15:33:30.207482 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194670 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 15:33:30.207482 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194673 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 15:33:30.207482 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.194676 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 15:33:30.207482 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.195294 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 15:33:30.207482 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.202838 2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 08 15:33:30.207482 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.202858 2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 08 15:33:30.207482 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.202886 2557 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 08 15:33:30.207482 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.202895 2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 08 15:33:30.207482 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.202987 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 15:33:30.207482 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.202995 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 15:33:30.207482 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203000 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 15:33:30.207482 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203005 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 15:33:30.207482 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203009 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 15:33:30.207482 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203014 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203018 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203023 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203027 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203032 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203036 2557 feature_gate.go:328] unrecognized feature gate: Example Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203042 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203046 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203050 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203055 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203059 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203064 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203068 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203073 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203080 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203087 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203093 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203098 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203104 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 15:33:30.207853 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203109 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203114 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203119 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203123 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203128 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203132 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203137 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203141 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203145 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203150 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203154 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203159 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203163 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203168 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203173 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203177 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203181 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203186 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203190 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203195 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 15:33:30.208441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203199 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203204 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203208 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203213 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203217 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203222 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203227 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203232 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203236 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203241 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203246 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203251 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203255 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203261 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203265 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203270 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203275 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203280 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203285 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 15:33:30.209073 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203290 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203294 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203299 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203303 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203308 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203312 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203316 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203320 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203325 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203330 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203334 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203339 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203343 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203347 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203352 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203356 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203360 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203365 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203369 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203373 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 15:33:30.209614 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203378 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 15:33:30.210126 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203383 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 15:33:30.210126 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203409 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 15:33:30.210126 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203415 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 15:33:30.210126 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203422 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 15:33:30.210126 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203427 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 15:33:30.210126 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203432 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 15:33:30.210126 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203438 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 15:33:30.210126 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203444 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 15:33:30.210126 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203451 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 15:33:30.210126 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203457 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 15:33:30.210126 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203482 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 15:33:30.210126 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203487 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 15:33:30.210126 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203492 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 15:33:30.210126 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203496 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 15:33:30.210126 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203500 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 15:33:30.210580 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.203509 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 15:33:30.210580 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203721 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 15:33:30.210580 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203729 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 15:33:30.210580 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203735 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 15:33:30.210580 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203740 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 15:33:30.210580 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203745 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 15:33:30.210580 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203750 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 15:33:30.210580 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203755 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 15:33:30.210580 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203760 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 15:33:30.210580 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203765 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 15:33:30.210580 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203770 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 15:33:30.210580 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203775 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 15:33:30.210580 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203780 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 15:33:30.210580 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203785 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203789 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203793 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203798 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203803 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203807 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203812 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203816 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203820 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203825 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203830 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203835 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203840 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203844 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203849 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203854 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203858 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203863 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203867 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203872 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 15:33:30.210924 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203876 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203881 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203886 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203890 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203894 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203898 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203902 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203907 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203911 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203916 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203920 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203924 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203929 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203934 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203938 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203943 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203948 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203952 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203957 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203961 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 15:33:30.211441 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203966 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203972 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203978 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203983 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203989 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203993 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.203998 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204002 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204007 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204011 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204016 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204021 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204025 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204030 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204034 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204039 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204044 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204048 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204052 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204057 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 15:33:30.212155 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204062 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204066 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204071 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204075 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204080 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204084 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204088 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204092 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204097 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204101 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204106 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204110 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204115 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204119 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204124 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204128 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204132 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204146 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204151 2557 feature_gate.go:328] unrecognized feature gate: Example Jun 08 15:33:30.213085 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204158 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 15:33:30.213861 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204164 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 15:33:30.213861 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204168 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 15:33:30.213861 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204175 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 15:33:30.213861 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204180 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 15:33:30.213861 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204186 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 15:33:30.213861 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204190 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 15:33:30.213861 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:30.204195 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 15:33:30.213861 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.204204 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 15:33:30.213861 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.204366 2557 server.go:962] "Client rotation is on, will bootstrap in background" Jun 08 15:33:30.213861 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.207942 2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 08 15:33:30.213861 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.209531 2557 server.go:1019] "Starting client certificate rotation" Jun 08 15:33:30.213861 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.209639 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 08 15:33:30.213861 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.209676 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 08 15:33:30.214200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.212434 2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 08 15:33:30.215830 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.215813 2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 08 15:33:30.226061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.226044 2557 log.go:25] "Validated CRI v1 runtime API" Jun 08 15:33:30.228553 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.228539 2557 log.go:25] "Validated CRI v1 image API" Jun 08 15:33:30.229347 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.229333 2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 08 15:33:30.231920 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.231902 2557 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a01fcaa7-efdb-4ab9-870a-f9493fb41a2d:/dev/nvme0n1p4 b068f6ec-eb10-407c-8738-c059bc00a507:/dev/nvme0n1p3] Jun 08 15:33:30.231986 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.231920 2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 08 15:33:30.236932 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.236821 2557 manager.go:216] Machine: {Timestamp:2026-06-08 15:33:30.235533916 +0000 UTC m=+0.324664957 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3151926 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2674e9af70383d8984b27bb47db462 SystemUUID:ec2674e9-af70-383d-8984-b27bb47db462 BootID:5614ccbd-5b40-447e-b418-7b998096662d Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:03:f8:1b:d5:4f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:03:f8:1b:d5:4f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:bf:58:00:f7:e6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 08 15:33:30.236932 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.236920 2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 08 15:33:30.237050 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.236990 2557 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 08 15:33:30.239848 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.239819 2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 08 15:33:30.239977 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.239850 2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-81.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 08 15:33:30.240567 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.240556 2557 topology_manager.go:138] "Creating topology manager with none policy" Jun 08 15:33:30.240598 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.240570 2557 container_manager_linux.go:309] "Creating device plugin manager" Jun 08 15:33:30.240598 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.240578 2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 08 15:33:30.241296 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.241287 2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 08 15:33:30.242073 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.242063 2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 08 15:33:30.242300 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.242280 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 08 15:33:30.245317 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.245306 2557 state_mem.go:36] "Initialized new in-memory state store" Jun 08 15:33:30.245432 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.245414 2557 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 08 15:33:30.245486 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.245451 2557 kubelet.go:486] "Attempting to sync node with API server" Jun 08 15:33:30.245486 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.245481 2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 08 15:33:30.245548 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.245503 2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 08 15:33:30.245548 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.245520 2557 kubelet.go:398] "Adding apiserver pod source" Jun 08 15:33:30.245548 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.245539 2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 08 15:33:30.245912 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.245899 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 08 15:33:30.245957 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.245923 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 08 15:33:30.249265 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.249250 2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 08 15:33:30.250569 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.250556 2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 08 15:33:30.250623 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.250587 2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 08 15:33:30.251121 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.251111 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 08 15:33:30.251156 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.251127 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 08 15:33:30.251156 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.251133 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 08 15:33:30.251156 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.251140 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 08 15:33:30.251156 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.251148 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 08 15:33:30.251156 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.251153 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 08 15:33:30.251283 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.251270 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 08 15:33:30.251283 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.251275 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 08 15:33:30.251283 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.251281 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 08 15:33:30.251364 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.251287 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 08 15:33:30.251364 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.251301 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 08 15:33:30.251364 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.251309 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 08 15:33:30.252171 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.252159 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 08 15:33:30.252171 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.252172 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 08 15:33:30.255181 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.255167 2557 server.go:1268] "Started kubelet" Jun 08 15:33:30.255270 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.255240 2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 08 15:33:30.255425 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.255370 2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 08 15:33:30.255550 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.255447 2557 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 08 15:33:30.258869 ip-10-0-134-81 systemd[1]: Started Kubernetes Kubelet. Jun 08 15:33:30.259210 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.259185 2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-81.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 08 15:33:30.259285 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.259228 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-81.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 08 15:33:30.259622 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.259564 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 08 15:33:30.260803 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.260629 2557 server.go:310] "Adding debug handlers to kubelet server" Jun 08 15:33:30.260890 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.260843 2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 08 15:33:30.262845 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.262824 2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 08 15:33:30.264715 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.263744 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-81.ec2.internal.18b724f4841885cd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-81.ec2.internal,UID:ip-10-0-134-81.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-81.ec2.internal,},FirstTimestamp:2026-06-08 15:33:30.255140301 +0000 UTC m=+0.344271341,LastTimestamp:2026-06-08 15:33:30.255140301 +0000 UTC m=+0.344271341,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-81.ec2.internal,}" Jun 08 15:33:30.266989 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.266973 2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 08 15:33:30.267066 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.266991 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 08 15:33:30.267066 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.267046 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-81.ec2.internal\" not found" Jun 08 15:33:30.268146 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.267510 2557 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 08 15:33:30.268146 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.267526 2557 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 08 15:33:30.268146 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.267605 2557 reconstruct.go:97] "Volume reconstruction finished" Jun 08 15:33:30.268146 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.267615 2557 reconciler.go:29] "Reconciler: start to sync state" Jun 08 15:33:30.268146 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.267719 2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 08 15:33:30.268146 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.268066 2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 08 15:33:30.268146 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.268077 2557 factory.go:55] Registering systemd factory Jun 08 15:33:30.268146 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.268087 2557 factory.go:223] Registration of the systemd container factory successfully Jun 08 15:33:30.268786 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.268740 2557 factory.go:153] Registering CRI-O factory Jun 08 15:33:30.268786 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.268753 2557 factory.go:223] Registration of the crio container factory successfully Jun 08 15:33:30.268786 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.268775 2557 factory.go:103] Registering Raw factory Jun 08 15:33:30.268926 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.268789 2557 manager.go:1196] Started watching for new ooms in manager Jun 08 15:33:30.269165 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.269149 2557 manager.go:349] Starting recovery of all containers Jun 08 15:33:30.277802 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.277642 2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-81.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 08 15:33:30.277898 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.277810 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 08 15:33:30.278928 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.278912 2557 manager.go:354] Recovery completed Jun 08 15:33:30.282935 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.282922 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 15:33:30.284627 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.284611 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasSufficientMemory" Jun 08 15:33:30.284713 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.284639 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 15:33:30.284713 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.284651 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasSufficientPID" Jun 08 15:33:30.285140 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.285127 2557 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 08 15:33:30.285140 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.285139 2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 08 15:33:30.285220 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.285154 2557 state_mem.go:36] "Initialized new in-memory state store" Jun 08 15:33:30.285220 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.285200 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 15:33:30.286230 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.286215 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasSufficientMemory" Jun 08 15:33:30.286300 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.286241 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 15:33:30.286300 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.286255 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasSufficientPID" Jun 08 15:33:30.286429 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.286418 2557 policy_none.go:49] "None policy: Start" Jun 08 15:33:30.286458 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.286433 2557 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 08 15:33:30.286458 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.286442 2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 08 15:33:30.287640 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.287629 2557 policy_none.go:47] "Start" Jun 08 15:33:30.291800 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.291784 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ghgvs" Jun 08 15:33:30.293574 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.293311 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-81.ec2.internal.18b724f485da71f8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-81.ec2.internal,UID:ip-10-0-134-81.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-81.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-81.ec2.internal,},FirstTimestamp:2026-06-08 15:33:30.284626424 +0000 UTC m=+0.373757465,LastTimestamp:2026-06-08 15:33:30.284626424 +0000 UTC m=+0.373757465,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-81.ec2.internal,}" Jun 08 15:33:30.296183 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.296123 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-81.ec2.internal.18b724f485dab8a2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-81.ec2.internal,UID:ip-10-0-134-81.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-81.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-81.ec2.internal,},FirstTimestamp:2026-06-08 15:33:30.284644514 +0000 UTC m=+0.373775554,LastTimestamp:2026-06-08 15:33:30.284644514 +0000 UTC m=+0.373775554,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-81.ec2.internal,}" Jun 08 15:33:30.298431 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.298414 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ghgvs" Jun 08 15:33:30.322004 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.321987 2557 manager.go:337] "Starting Device Plugin manager" Jun 08 15:33:30.333318 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.322021 2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 08 15:33:30.333318 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.322035 2557 server.go:87] "Starting device plugin registration server" Jun 08 15:33:30.333318 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.322251 2557 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 08 15:33:30.333318 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.322263 2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 08 15:33:30.333318 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.322353 2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 08 15:33:30.333318 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.322429 2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 08 15:33:30.333318 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.322436 2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 08 15:33:30.333318 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.322993 2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 08 15:33:30.333318 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.323041 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-81.ec2.internal\" not found" Jun 08 15:33:30.333318 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.330364 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 08 15:33:30.333318 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.331552 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 08 15:33:30.333318 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.331586 2557 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 08 15:33:30.333318 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.331607 2557 kubelet.go:2448] "Starting kubelet main sync loop" Jun 08 15:33:30.333318 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.331639 2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 08 15:33:30.333915 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.333898 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 15:33:30.422885 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.422841 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 15:33:30.424475 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.424447 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasSufficientMemory" Jun 08 15:33:30.424554 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.424496 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 15:33:30.424554 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.424508 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasSufficientPID" Jun 08 15:33:30.424554 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.424528 2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.432261 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.432244 2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-81.ec2.internal"] Jun 08 15:33:30.432318 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.432298 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 15:33:30.433071 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.433056 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasSufficientMemory" Jun 08 15:33:30.433132 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.433082 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 15:33:30.433132 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.433092 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasSufficientPID" Jun 08 15:33:30.433653 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.433640 2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.433694 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.433658 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-81.ec2.internal\": node \"ip-10-0-134-81.ec2.internal\" not found" Jun 08 15:33:30.434370 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.434359 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 15:33:30.434500 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.434487 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.434539 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.434516 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 15:33:30.435032 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.435002 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasSufficientMemory" Jun 08 15:33:30.435032 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.435022 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 15:33:30.435032 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.435031 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasSufficientPID" Jun 08 15:33:30.435198 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.435050 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasSufficientMemory" Jun 08 15:33:30.435198 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.435069 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 15:33:30.435198 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.435079 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasSufficientPID" Jun 08 15:33:30.436319 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.436302 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.436375 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.436337 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 15:33:30.437014 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.436998 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasSufficientMemory" Jun 08 15:33:30.437106 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.437026 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 15:33:30.437106 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.437038 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasSufficientPID" Jun 08 15:33:30.450964 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.450944 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-81.ec2.internal\" not found" node="ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.455218 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.455200 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-81.ec2.internal\" not found" node="ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.455284 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.455276 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-81.ec2.internal\" not found" Jun 08 15:33:30.555760 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.555737 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-81.ec2.internal\" not found" Jun 08 15:33:30.569031 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.569014 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0d691a0e751fe5c9ef931276cec8b51d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-81.ec2.internal\" (UID: \"0d691a0e751fe5c9ef931276cec8b51d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.569083 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.569038 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/35c8ede3c284d0621d6a95a5857fea78-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal\" (UID: \"35c8ede3c284d0621d6a95a5857fea78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.569083 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.569055 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/35c8ede3c284d0621d6a95a5857fea78-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal\" (UID: \"35c8ede3c284d0621d6a95a5857fea78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.656175 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.656143 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-81.ec2.internal\" not found" Jun 08 15:33:30.669589 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.669543 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/35c8ede3c284d0621d6a95a5857fea78-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal\" (UID: \"35c8ede3c284d0621d6a95a5857fea78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.669589 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.669566 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/35c8ede3c284d0621d6a95a5857fea78-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal\" (UID: \"35c8ede3c284d0621d6a95a5857fea78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.669665 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.669577 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0d691a0e751fe5c9ef931276cec8b51d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-81.ec2.internal\" (UID: \"0d691a0e751fe5c9ef931276cec8b51d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.669665 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.669633 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/35c8ede3c284d0621d6a95a5857fea78-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal\" (UID: \"35c8ede3c284d0621d6a95a5857fea78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.669665 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.669607 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0d691a0e751fe5c9ef931276cec8b51d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-81.ec2.internal\" (UID: \"0d691a0e751fe5c9ef931276cec8b51d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.669754 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.669662 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/35c8ede3c284d0621d6a95a5857fea78-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal\" (UID: \"35c8ede3c284d0621d6a95a5857fea78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.755930 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.755898 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.756986 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.756972 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-81.ec2.internal\" not found" Jun 08 15:33:30.759214 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:30.759107 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-81.ec2.internal" Jun 08 15:33:30.857312 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.857281 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-81.ec2.internal\" not found" Jun 08 15:33:30.957896 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:30.957827 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-81.ec2.internal\" not found" Jun 08 15:33:31.058330 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:31.058282 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-81.ec2.internal\" not found" Jun 08 15:33:31.158872 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:31.158835 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-81.ec2.internal\" not found" Jun 08 15:33:31.209336 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.209272 2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 08 15:33:31.209990 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.209486 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 08 15:33:31.259750 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:31.259722 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-81.ec2.internal\" not found" Jun 08 15:33:31.276121 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:31.276094 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c8ede3c284d0621d6a95a5857fea78.slice/crio-d9a032ce2a40a2bc04542d1c94a8d2500f77fb263855d1191d9be1154e52c969 WatchSource:0}: Error finding container d9a032ce2a40a2bc04542d1c94a8d2500f77fb263855d1191d9be1154e52c969: Status 404 returned error can't find the container with id d9a032ce2a40a2bc04542d1c94a8d2500f77fb263855d1191d9be1154e52c969 Jun 08 15:33:31.276599 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:31.276579 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d691a0e751fe5c9ef931276cec8b51d.slice/crio-485fbcdb351f88e470286249b5364e03d9290694a34dbf7d529be00661a27a81 WatchSource:0}: Error finding container 485fbcdb351f88e470286249b5364e03d9290694a34dbf7d529be00661a27a81: Status 404 returned error can't find the container with id 485fbcdb351f88e470286249b5364e03d9290694a34dbf7d529be00661a27a81 Jun 08 15:33:31.281895 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.281882 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 15:33:31.300976 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.300948 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-07 15:28:30 +0000 UTC" deadline="2028-01-09 16:32:10.58821579 +0000 UTC" Jun 08 15:33:31.301051 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.300976 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13920h58m39.287244023s" Jun 08 15:33:31.334180 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.334139 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-81.ec2.internal" event={"ID":"0d691a0e751fe5c9ef931276cec8b51d","Type":"ContainerStarted","Data":"485fbcdb351f88e470286249b5364e03d9290694a34dbf7d529be00661a27a81"} Jun 08 15:33:31.335148 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.335122 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal" event={"ID":"35c8ede3c284d0621d6a95a5857fea78","Type":"ContainerStarted","Data":"d9a032ce2a40a2bc04542d1c94a8d2500f77fb263855d1191d9be1154e52c969"} Jun 08 15:33:31.360278 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:31.360257 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-81.ec2.internal\" not found" Jun 08 15:33:31.414806 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.414787 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 15:33:31.416092 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.416074 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasSufficientMemory" Jun 08 15:33:31.416169 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.416117 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 15:33:31.416169 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.416128 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeHasSufficientPID" Jun 08 15:33:31.460499 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:31.460433 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-81.ec2.internal\" not found" Jun 08 15:33:31.515008 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.514984 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 15:33:31.560541 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:31.560506 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-81.ec2.internal\" not found" Jun 08 15:33:31.661087 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:31.661052 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-81.ec2.internal\" not found" Jun 08 15:33:31.724691 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.724623 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 15:33:31.729101 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.729070 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 15:33:31.767964 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.767940 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal" Jun 08 15:33:31.779168 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.779149 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 08 15:33:31.780021 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.780006 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-81.ec2.internal" Jun 08 15:33:31.787791 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:31.787767 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 08 15:33:32.247336 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.247309 2557 apiserver.go:52] "Watching apiserver" Jun 08 15:33:32.253907 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.253885 2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 08 15:33:32.254294 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.254272 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-81.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6","openshift-image-registry/node-ca-5t7fb","openshift-multus/multus-qmt5n","openshift-multus/network-metrics-daemon-hct6z","openshift-network-diagnostics/network-check-target-mvb2c","openshift-cluster-node-tuning-operator/tuned-8x928","openshift-dns/node-resolver-l8rfj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal","openshift-multus/multus-additional-cni-plugins-tpwvw","openshift-network-operator/iptables-alerter-99v6k","openshift-ovn-kubernetes/ovnkube-node-tsb88","kube-system/global-pull-secret-syncer-r2mpb","kube-system/konnectivity-agent-8gf2f"] Jun 08 15:33:32.256277 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.256248 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.257647 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.257628 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:32.257733 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.257714 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:33:32.258491 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.258454 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 08 15:33:32.258594 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.258542 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 08 15:33:32.258594 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.258562 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cbr48\"" Jun 08 15:33:32.259195 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.258903 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5t7fb" Jun 08 15:33:32.260522 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.260499 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.261243 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.261223 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 08 15:33:32.261361 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.261340 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 08 15:33:32.261524 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.261506 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 08 15:33:32.261592 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.261543 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5m262\"" Jun 08 15:33:32.261773 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.261756 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:32.261842 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.261821 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:33:32.262369 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.262351 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 08 15:33:32.262453 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.262375 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 08 15:33:32.262904 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.262620 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 08 15:33:32.262904 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.262863 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:32.263046 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.262924 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:33:32.263300 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.263280 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 08 15:33:32.263372 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.263356 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vg4gg\"" Jun 08 15:33:32.266807 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.266732 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8gf2f" Jun 08 15:33:32.267532 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.267163 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 08 15:33:32.268047 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.268019 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l8rfj" Jun 08 15:33:32.269566 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.269447 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 08 15:33:32.269566 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.269450 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t4zn7\"" Jun 08 15:33:32.269566 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.269532 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 08 15:33:32.270292 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.269926 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.271521 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.271260 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 08 15:33:32.271521 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.271243 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.271521 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.271334 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 08 15:33:32.271678 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.271557 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-s597t\"" Jun 08 15:33:32.272988 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.272449 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 08 15:33:32.272988 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.272519 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 08 15:33:32.272988 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.272655 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 08 15:33:32.272988 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.272881 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nh9wl\"" Jun 08 15:33:32.273315 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.273302 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7nh4h\"" Jun 08 15:33:32.273478 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.273444 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 08 15:33:32.273786 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.273766 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-99v6k" Jun 08 15:33:32.276558 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.276250 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 08 15:33:32.276558 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.276320 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 08 15:33:32.276558 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.276340 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.276558 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.276322 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 08 15:33:32.276558 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.276396 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 08 15:33:32.276558 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.276527 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l9rwt\"" Jun 08 15:33:32.278488 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.278454 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-sysconfig\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.279035 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.278793 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 08 15:33:32.279118 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279074 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 08 15:33:32.279348 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279009 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-sysctl-d\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.279348 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279258 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-sysctl-conf\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.279348 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279283 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-cnibin\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.279348 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279309 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1e9d36fd-7367-43bf-8a04-3675efc9957d-kubelet-config\") pod \"global-pull-secret-syncer-r2mpb\" (UID: \"1e9d36fd-7367-43bf-8a04-3675efc9957d\") " pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:32.279574 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279369 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1e9d36fd-7367-43bf-8a04-3675efc9957d-dbus\") pod \"global-pull-secret-syncer-r2mpb\" (UID: \"1e9d36fd-7367-43bf-8a04-3675efc9957d\") " pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:32.279574 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279400 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-registration-dir\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.279574 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279457 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kx84b\"" Jun 08 15:33:32.279574 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279506 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-etc-kubernetes\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.279574 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279541 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9865e636-5527-48fe-b4a8-de43e6b507ec-tmp-dir\") pod \"node-resolver-l8rfj\" (UID: \"9865e636-5527-48fe-b4a8-de43e6b507ec\") " pod="openshift-dns/node-resolver-l8rfj" Jun 08 15:33:32.279574 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279548 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 08 15:33:32.279845 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279583 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkvwc\" (UniqueName: \"kubernetes.io/projected/9865e636-5527-48fe-b4a8-de43e6b507ec-kube-api-access-kkvwc\") pod \"node-resolver-l8rfj\" (UID: \"9865e636-5527-48fe-b4a8-de43e6b507ec\") " pod="openshift-dns/node-resolver-l8rfj" Jun 08 15:33:32.279845 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279603 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.279845 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279617 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-var-lib-kubelet\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.279845 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279633 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-tuned\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.279845 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279672 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e3cc3eb-f4c6-416c-bcb8-677729009da4-tmp\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.279845 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279688 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2wnm\" (UniqueName: \"kubernetes.io/projected/1e3cc3eb-f4c6-416c-bcb8-677729009da4-kube-api-access-q2wnm\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.279845 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279695 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 08 15:33:32.279845 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279710 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-multus-cni-dir\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.279845 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279736 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-cni-binary-copy\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.279845 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279753 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-run\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.279845 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279790 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-host\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.279845 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279844 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-socket-dir\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279874 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-multus-socket-dir-parent\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279915 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-run-netns\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279950 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-var-lib-cni-multus\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.279981 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-multus-conf-dir\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280010 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs\") pod \"network-metrics-daemon-hct6z\" (UID: \"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58\") " pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280014 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280035 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5krxh\" (UniqueName: \"kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh\") pod \"network-check-target-mvb2c\" (UID: \"7711bd0d-d822-4d5d-b62f-84f831446da0\") " pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280066 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-sys-fs\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280085 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280092 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret\") pod \"global-pull-secret-syncer-r2mpb\" (UID: \"1e9d36fd-7367-43bf-8a04-3675efc9957d\") " pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280117 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-etc-selinux\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280142 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh7rq\" (UniqueName: \"kubernetes.io/projected/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-kube-api-access-kh7rq\") pod \"network-metrics-daemon-hct6z\" (UID: \"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58\") " pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280172 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-systemd\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280202 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-lib-modules\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280249 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-device-dir\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280274 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d777d945-3cc7-4e04-b64b-f021f129aea7-host\") pod \"node-ca-5t7fb\" (UID: \"d777d945-3cc7-4e04-b64b-f021f129aea7\") " pod="openshift-image-registry/node-ca-5t7fb" Jun 08 15:33:32.280373 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280307 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfpwr\" (UniqueName: \"kubernetes.io/projected/d777d945-3cc7-4e04-b64b-f021f129aea7-kube-api-access-xfpwr\") pod \"node-ca-5t7fb\" (UID: \"d777d945-3cc7-4e04-b64b-f021f129aea7\") " pod="openshift-image-registry/node-ca-5t7fb" Jun 08 15:33:32.281126 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280332 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-os-release\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.281126 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280361 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-var-lib-cni-bin\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.281126 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280385 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-multus-daemon-config\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.281126 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280408 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-run-multus-certs\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.281126 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.280430 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-sys\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.281722 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.281702 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s66b6\" (UniqueName: \"kubernetes.io/projected/9b749aee-cd21-478f-92b7-544e0df2045e-kube-api-access-s66b6\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.282230 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.282049 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d777d945-3cc7-4e04-b64b-f021f129aea7-serviceca\") pod \"node-ca-5t7fb\" (UID: \"d777d945-3cc7-4e04-b64b-f021f129aea7\") " pod="openshift-image-registry/node-ca-5t7fb" Jun 08 15:33:32.282230 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.282108 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-run-k8s-cni-cncf-io\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.282230 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.282135 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-hostroot\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.282230 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.282158 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kptw\" (UniqueName: \"kubernetes.io/projected/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-kube-api-access-9kptw\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.282530 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.282255 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-modprobe-d\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.282530 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.282357 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-kubernetes\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.282530 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.282382 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-system-cni-dir\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.282530 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.282423 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-var-lib-kubelet\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.282530 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.282449 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff5279a7-eec2-4e1b-a38f-ebee10911dd1-agent-certs\") pod \"konnectivity-agent-8gf2f\" (UID: \"ff5279a7-eec2-4e1b-a38f-ebee10911dd1\") " pod="kube-system/konnectivity-agent-8gf2f" Jun 08 15:33:32.282745 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.282529 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff5279a7-eec2-4e1b-a38f-ebee10911dd1-konnectivity-ca\") pod \"konnectivity-agent-8gf2f\" (UID: \"ff5279a7-eec2-4e1b-a38f-ebee10911dd1\") " pod="kube-system/konnectivity-agent-8gf2f" Jun 08 15:33:32.282745 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.282635 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9865e636-5527-48fe-b4a8-de43e6b507ec-hosts-file\") pod \"node-resolver-l8rfj\" (UID: \"9865e636-5527-48fe-b4a8-de43e6b507ec\") " pod="openshift-dns/node-resolver-l8rfj" Jun 08 15:33:32.284456 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.283942 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 08 15:33:32.302215 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.302182 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wkvq2" Jun 08 15:33:32.307747 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.307729 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wkvq2" Jun 08 15:33:32.368713 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.368692 2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 08 15:33:32.382871 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.382846 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-device-dir\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.383003 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.382881 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-os-release\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.383003 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.382899 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-multus-daemon-config\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.383003 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.382916 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-sys\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.383003 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.382935 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-cnibin\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.383003 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.382961 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s66b6\" (UniqueName: \"kubernetes.io/projected/9b749aee-cd21-478f-92b7-544e0df2045e-kube-api-access-s66b6\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.383003 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.382984 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-device-dir\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.383274 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383014 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-os-release\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.383274 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383015 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-sys\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.383274 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383061 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d777d945-3cc7-4e04-b64b-f021f129aea7-serviceca\") pod \"node-ca-5t7fb\" (UID: \"d777d945-3cc7-4e04-b64b-f021f129aea7\") " pod="openshift-image-registry/node-ca-5t7fb" Jun 08 15:33:32.383274 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383119 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-run-k8s-cni-cncf-io\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.383274 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383166 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.383274 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383207 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-run-k8s-cni-cncf-io\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.383274 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383261 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383291 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-system-cni-dir\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383316 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-var-lib-kubelet\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383340 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff5279a7-eec2-4e1b-a38f-ebee10911dd1-konnectivity-ca\") pod \"konnectivity-agent-8gf2f\" (UID: \"ff5279a7-eec2-4e1b-a38f-ebee10911dd1\") " pod="kube-system/konnectivity-agent-8gf2f" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383363 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9865e636-5527-48fe-b4a8-de43e6b507ec-hosts-file\") pod \"node-resolver-l8rfj\" (UID: \"9865e636-5527-48fe-b4a8-de43e6b507ec\") " pod="openshift-dns/node-resolver-l8rfj" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383387 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-slash\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383403 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-system-cni-dir\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383406 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-var-lib-kubelet\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383426 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-multus-daemon-config\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383415 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3be431e0-395c-47b8-a975-37cf4b23933f-iptables-alerter-script\") pod \"iptables-alerter-99v6k\" (UID: \"3be431e0-395c-47b8-a975-37cf4b23933f\") " pod="openshift-network-operator/iptables-alerter-99v6k" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383478 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9865e636-5527-48fe-b4a8-de43e6b507ec-hosts-file\") pod \"node-resolver-l8rfj\" (UID: \"9865e636-5527-48fe-b4a8-de43e6b507ec\") " pod="openshift-dns/node-resolver-l8rfj" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383494 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1e9d36fd-7367-43bf-8a04-3675efc9957d-kubelet-config\") pod \"global-pull-secret-syncer-r2mpb\" (UID: \"1e9d36fd-7367-43bf-8a04-3675efc9957d\") " pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383519 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-etc-kubernetes\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383522 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d777d945-3cc7-4e04-b64b-f021f129aea7-serviceca\") pod \"node-ca-5t7fb\" (UID: \"d777d945-3cc7-4e04-b64b-f021f129aea7\") " pod="openshift-image-registry/node-ca-5t7fb" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383534 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9865e636-5527-48fe-b4a8-de43e6b507ec-tmp-dir\") pod \"node-resolver-l8rfj\" (UID: \"9865e636-5527-48fe-b4a8-de43e6b507ec\") " pod="openshift-dns/node-resolver-l8rfj" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383551 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkvwc\" (UniqueName: \"kubernetes.io/projected/9865e636-5527-48fe-b4a8-de43e6b507ec-kube-api-access-kkvwc\") pod \"node-resolver-l8rfj\" (UID: \"9865e636-5527-48fe-b4a8-de43e6b507ec\") " pod="openshift-dns/node-resolver-l8rfj" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383585 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1e9d36fd-7367-43bf-8a04-3675efc9957d-kubelet-config\") pod \"global-pull-secret-syncer-r2mpb\" (UID: \"1e9d36fd-7367-43bf-8a04-3675efc9957d\") " pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:32.383581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383586 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-var-lib-kubelet\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383602 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-etc-kubernetes\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383613 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-tuned\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383629 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-var-lib-kubelet\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383646 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383686 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-run-ovn-kubernetes\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383714 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-var-lib-cni-multus\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383738 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-multus-conf-dir\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383763 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5krxh\" (UniqueName: \"kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh\") pod \"network-check-target-mvb2c\" (UID: \"7711bd0d-d822-4d5d-b62f-84f831446da0\") " pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383788 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5v4d\" (UniqueName: \"kubernetes.io/projected/3be431e0-395c-47b8-a975-37cf4b23933f-kube-api-access-z5v4d\") pod \"iptables-alerter-99v6k\" (UID: \"3be431e0-395c-47b8-a975-37cf4b23933f\") " pod="openshift-network-operator/iptables-alerter-99v6k" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383796 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-var-lib-cni-multus\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383813 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-systemd-units\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383839 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-etc-openvswitch\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383862 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-log-socket\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383888 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k2fq\" (UniqueName: \"kubernetes.io/projected/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-kube-api-access-5k2fq\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383884 2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383915 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh7rq\" (UniqueName: \"kubernetes.io/projected/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-kube-api-access-kh7rq\") pod \"network-metrics-daemon-hct6z\" (UID: \"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58\") " pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:32.384376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383941 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-lib-modules\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383943 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9865e636-5527-48fe-b4a8-de43e6b507ec-tmp-dir\") pod \"node-resolver-l8rfj\" (UID: \"9865e636-5527-48fe-b4a8-de43e6b507ec\") " pod="openshift-dns/node-resolver-l8rfj" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383961 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff5279a7-eec2-4e1b-a38f-ebee10911dd1-konnectivity-ca\") pod \"konnectivity-agent-8gf2f\" (UID: \"ff5279a7-eec2-4e1b-a38f-ebee10911dd1\") " pod="kube-system/konnectivity-agent-8gf2f" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.383966 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-node-log\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384033 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-multus-conf-dir\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384038 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d777d945-3cc7-4e04-b64b-f021f129aea7-host\") pod \"node-ca-5t7fb\" (UID: \"d777d945-3cc7-4e04-b64b-f021f129aea7\") " pod="openshift-image-registry/node-ca-5t7fb" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384086 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfpwr\" (UniqueName: \"kubernetes.io/projected/d777d945-3cc7-4e04-b64b-f021f129aea7-kube-api-access-xfpwr\") pod \"node-ca-5t7fb\" (UID: \"d777d945-3cc7-4e04-b64b-f021f129aea7\") " pod="openshift-image-registry/node-ca-5t7fb" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384101 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-lib-modules\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384128 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d777d945-3cc7-4e04-b64b-f021f129aea7-host\") pod \"node-ca-5t7fb\" (UID: \"d777d945-3cc7-4e04-b64b-f021f129aea7\") " pod="openshift-image-registry/node-ca-5t7fb" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384139 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-var-lib-cni-bin\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384233 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-var-lib-cni-bin\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384263 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-run-multus-certs\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384310 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-cni-netd\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384319 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-run-multus-certs\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384340 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-hostroot\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384354 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kptw\" (UniqueName: \"kubernetes.io/projected/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-kube-api-access-9kptw\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384368 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-modprobe-d\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384383 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-kubernetes\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.385162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384406 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-hostroot\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384413 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htnhp\" (UniqueName: \"kubernetes.io/projected/efff1108-d45f-4dd8-9cf8-24cbbd122b87-kube-api-access-htnhp\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384450 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-os-release\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384485 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-kubernetes\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384499 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-modprobe-d\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384515 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-run\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384535 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-host\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384558 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff5279a7-eec2-4e1b-a38f-ebee10911dd1-agent-certs\") pod \"konnectivity-agent-8gf2f\" (UID: \"ff5279a7-eec2-4e1b-a38f-ebee10911dd1\") " pod="kube-system/konnectivity-agent-8gf2f" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384599 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-run\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384612 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-host\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384656 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-sysconfig\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384697 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-sysctl-d\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384714 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-sysconfig\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384721 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-sysctl-conf\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384750 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e3cc3eb-f4c6-416c-bcb8-677729009da4-tmp\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384775 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-cnibin\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384798 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1e9d36fd-7367-43bf-8a04-3675efc9957d-dbus\") pod \"global-pull-secret-syncer-r2mpb\" (UID: \"1e9d36fd-7367-43bf-8a04-3675efc9957d\") " pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384823 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-registration-dir\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.386000 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384832 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-sysctl-conf\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384849 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3be431e0-395c-47b8-a975-37cf4b23933f-host-slash\") pod \"iptables-alerter-99v6k\" (UID: \"3be431e0-395c-47b8-a975-37cf4b23933f\") " pod="openshift-network-operator/iptables-alerter-99v6k" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384866 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-sysctl-d\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384886 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-cnibin\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384872 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-kubelet\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384920 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-run-systemd\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384933 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-registration-dir\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384944 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-var-lib-openvswitch\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384966 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1e9d36fd-7367-43bf-8a04-3675efc9957d-dbus\") pod \"global-pull-secret-syncer-r2mpb\" (UID: \"1e9d36fd-7367-43bf-8a04-3675efc9957d\") " pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.384986 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385011 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2wnm\" (UniqueName: \"kubernetes.io/projected/1e3cc3eb-f4c6-416c-bcb8-677729009da4-kube-api-access-q2wnm\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385037 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/efff1108-d45f-4dd8-9cf8-24cbbd122b87-ovnkube-config\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385043 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385059 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/efff1108-d45f-4dd8-9cf8-24cbbd122b87-env-overrides\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385093 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-system-cni-dir\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385127 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385157 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-multus-cni-dir\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.386513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385179 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-cni-binary-copy\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385202 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/efff1108-d45f-4dd8-9cf8-24cbbd122b87-ovn-node-metrics-cert\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385226 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/efff1108-d45f-4dd8-9cf8-24cbbd122b87-ovnkube-script-lib\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385234 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-multus-cni-dir\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385249 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-cni-binary-copy\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385275 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-socket-dir\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385298 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-multus-socket-dir-parent\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385322 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-run-netns\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385343 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs\") pod \"network-metrics-daemon-hct6z\" (UID: \"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58\") " pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385365 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-run-openvswitch\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385389 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-run-ovn\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385412 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-cni-bin\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385455 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-sys-fs\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385501 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret\") pod \"global-pull-secret-syncer-r2mpb\" (UID: \"1e9d36fd-7367-43bf-8a04-3675efc9957d\") " pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.385521 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385526 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-etc-selinux\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385552 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-systemd\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.387061 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.385587 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs podName:75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58 nodeName:}" failed. No retries permitted until 2026-06-08 15:33:32.885569806 +0000 UTC m=+2.974700838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs") pod "network-metrics-daemon-hct6z" (UID: "75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 15:33:32.387877 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385605 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-run-netns\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.387877 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385622 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-systemd\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.387877 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385415 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-socket-dir\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.387877 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385704 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-cni-binary-copy\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.387877 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.385733 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 15:33:32.387877 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385741 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-multus-socket-dir-parent\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.387877 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385777 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-host-run-netns\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.387877 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.385798 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret podName:1e9d36fd-7367-43bf-8a04-3675efc9957d nodeName:}" failed. No retries permitted until 2026-06-08 15:33:32.885778587 +0000 UTC m=+2.974909618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret") pod "global-pull-secret-syncer-r2mpb" (UID: "1e9d36fd-7367-43bf-8a04-3675efc9957d") : object "kube-system"/"original-pull-secret" not registered Jun 08 15:33:32.387877 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385852 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-sys-fs\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.387877 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.385882 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9b749aee-cd21-478f-92b7-544e0df2045e-etc-selinux\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.387877 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.387160 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e3cc3eb-f4c6-416c-bcb8-677729009da4-tmp\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.387877 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.387234 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e3cc3eb-f4c6-416c-bcb8-677729009da4-etc-tuned\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.387877 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.387585 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff5279a7-eec2-4e1b-a38f-ebee10911dd1-agent-certs\") pod \"konnectivity-agent-8gf2f\" (UID: \"ff5279a7-eec2-4e1b-a38f-ebee10911dd1\") " pod="kube-system/konnectivity-agent-8gf2f" Jun 08 15:33:32.391686 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.391659 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 15:33:32.391791 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.391695 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 15:33:32.391791 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.391711 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5krxh for pod openshift-network-diagnostics/network-check-target-mvb2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 15:33:32.391791 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.391778 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh podName:7711bd0d-d822-4d5d-b62f-84f831446da0 nodeName:}" failed. No retries permitted until 2026-06-08 15:33:32.891760973 +0000 UTC m=+2.980892025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5krxh" (UniqueName: "kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh") pod "network-check-target-mvb2c" (UID: "7711bd0d-d822-4d5d-b62f-84f831446da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 15:33:32.393180 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.392732 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkvwc\" (UniqueName: \"kubernetes.io/projected/9865e636-5527-48fe-b4a8-de43e6b507ec-kube-api-access-kkvwc\") pod \"node-resolver-l8rfj\" (UID: \"9865e636-5527-48fe-b4a8-de43e6b507ec\") " pod="openshift-dns/node-resolver-l8rfj" Jun 08 15:33:32.393180 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.393138 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfpwr\" (UniqueName: \"kubernetes.io/projected/d777d945-3cc7-4e04-b64b-f021f129aea7-kube-api-access-xfpwr\") pod \"node-ca-5t7fb\" (UID: \"d777d945-3cc7-4e04-b64b-f021f129aea7\") " pod="openshift-image-registry/node-ca-5t7fb" Jun 08 15:33:32.393321 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.393253 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s66b6\" (UniqueName: \"kubernetes.io/projected/9b749aee-cd21-478f-92b7-544e0df2045e-kube-api-access-s66b6\") pod \"aws-ebs-csi-driver-node-hsnm6\" (UID: \"9b749aee-cd21-478f-92b7-544e0df2045e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.393321 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.393283 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kptw\" (UniqueName: \"kubernetes.io/projected/6a8b4c5c-e2c8-4a49-87b2-e80996d54193-kube-api-access-9kptw\") pod \"multus-qmt5n\" (UID: \"6a8b4c5c-e2c8-4a49-87b2-e80996d54193\") " pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.393614 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.393584 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh7rq\" (UniqueName: \"kubernetes.io/projected/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-kube-api-access-kh7rq\") pod \"network-metrics-daemon-hct6z\" (UID: \"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58\") " pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:32.395150 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.395130 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2wnm\" (UniqueName: \"kubernetes.io/projected/1e3cc3eb-f4c6-416c-bcb8-677729009da4-kube-api-access-q2wnm\") pod \"tuned-8x928\" (UID: \"1e3cc3eb-f4c6-416c-bcb8-677729009da4\") " pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.485995 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.485965 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htnhp\" (UniqueName: \"kubernetes.io/projected/efff1108-d45f-4dd8-9cf8-24cbbd122b87-kube-api-access-htnhp\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.485995 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.485997 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-os-release\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.486216 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486021 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3be431e0-395c-47b8-a975-37cf4b23933f-host-slash\") pod \"iptables-alerter-99v6k\" (UID: \"3be431e0-395c-47b8-a975-37cf4b23933f\") " pod="openshift-network-operator/iptables-alerter-99v6k" Jun 08 15:33:32.486216 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486064 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-kubelet\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486216 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486127 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-run-systemd\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486216 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486133 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-os-release\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.486216 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486151 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-var-lib-openvswitch\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486216 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486166 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-kubelet\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486216 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486187 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/efff1108-d45f-4dd8-9cf8-24cbbd122b87-ovnkube-config\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486216 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486193 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-var-lib-openvswitch\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486216 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486173 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3be431e0-395c-47b8-a975-37cf4b23933f-host-slash\") pod \"iptables-alerter-99v6k\" (UID: \"3be431e0-395c-47b8-a975-37cf4b23933f\") " pod="openshift-network-operator/iptables-alerter-99v6k" Jun 08 15:33:32.486216 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486214 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-run-systemd\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486227 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/efff1108-d45f-4dd8-9cf8-24cbbd122b87-env-overrides\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486263 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-system-cni-dir\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486283 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486302 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/efff1108-d45f-4dd8-9cf8-24cbbd122b87-ovn-node-metrics-cert\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486334 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/efff1108-d45f-4dd8-9cf8-24cbbd122b87-ovnkube-script-lib\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486342 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-system-cni-dir\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486449 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-cni-binary-copy\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486514 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-run-openvswitch\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486539 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-run-ovn\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486562 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-cni-bin\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486603 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-run-netns\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486616 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486630 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-cnibin\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486650 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486660 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-cni-bin\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486620 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-run-openvswitch\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.486682 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486668 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486694 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-cnibin\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486662 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-run-ovn\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486699 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/efff1108-d45f-4dd8-9cf8-24cbbd122b87-env-overrides\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486752 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-run-netns\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486762 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486770 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/efff1108-d45f-4dd8-9cf8-24cbbd122b87-ovnkube-config\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486775 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-slash\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486807 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-slash\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486820 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3be431e0-395c-47b8-a975-37cf4b23933f-iptables-alerter-script\") pod \"iptables-alerter-99v6k\" (UID: \"3be431e0-395c-47b8-a975-37cf4b23933f\") " pod="openshift-network-operator/iptables-alerter-99v6k" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486854 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486882 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-run-ovn-kubernetes\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486920 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5v4d\" (UniqueName: \"kubernetes.io/projected/3be431e0-395c-47b8-a975-37cf4b23933f-kube-api-access-z5v4d\") pod \"iptables-alerter-99v6k\" (UID: \"3be431e0-395c-47b8-a975-37cf4b23933f\") " pod="openshift-network-operator/iptables-alerter-99v6k" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486964 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-systemd-units\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486990 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-etc-openvswitch\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.486998 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-run-ovn-kubernetes\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.487007 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-cni-binary-copy\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.487519 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.487013 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-log-socket\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.488167 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.487057 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-etc-openvswitch\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.488167 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.487076 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k2fq\" (UniqueName: \"kubernetes.io/projected/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-kube-api-access-5k2fq\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.488167 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.487099 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-systemd-units\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.488167 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.487106 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-node-log\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.488167 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.487136 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-log-socket\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.488167 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.487136 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-cni-netd\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.488167 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.487169 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-host-cni-netd\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.488167 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.487173 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.488167 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.487202 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/efff1108-d45f-4dd8-9cf8-24cbbd122b87-node-log\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.488167 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.487211 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/efff1108-d45f-4dd8-9cf8-24cbbd122b87-ovnkube-script-lib\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.488167 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.487494 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3be431e0-395c-47b8-a975-37cf4b23933f-iptables-alerter-script\") pod \"iptables-alerter-99v6k\" (UID: \"3be431e0-395c-47b8-a975-37cf4b23933f\") " pod="openshift-network-operator/iptables-alerter-99v6k" Jun 08 15:33:32.488167 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.487614 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.489186 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.489164 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/efff1108-d45f-4dd8-9cf8-24cbbd122b87-ovn-node-metrics-cert\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.494065 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.494045 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htnhp\" (UniqueName: \"kubernetes.io/projected/efff1108-d45f-4dd8-9cf8-24cbbd122b87-kube-api-access-htnhp\") pod \"ovnkube-node-tsb88\" (UID: \"efff1108-d45f-4dd8-9cf8-24cbbd122b87\") " pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.494272 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.494252 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k2fq\" (UniqueName: \"kubernetes.io/projected/a7cad45c-9b89-43d6-bd57-7b3194b1d7c3-kube-api-access-5k2fq\") pod \"multus-additional-cni-plugins-tpwvw\" (UID: \"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3\") " pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.494598 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.494578 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5v4d\" (UniqueName: \"kubernetes.io/projected/3be431e0-395c-47b8-a975-37cf4b23933f-kube-api-access-z5v4d\") pod \"iptables-alerter-99v6k\" (UID: \"3be431e0-395c-47b8-a975-37cf4b23933f\") " pod="openshift-network-operator/iptables-alerter-99v6k" Jun 08 15:33:32.534229 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.534163 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 15:33:32.570787 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.570761 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8x928" Jun 08 15:33:32.578503 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.578456 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5t7fb" Jun 08 15:33:32.589365 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.589334 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qmt5n" Jun 08 15:33:32.595931 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.595913 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8gf2f" Jun 08 15:33:32.603697 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.603678 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l8rfj" Jun 08 15:33:32.611311 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.611294 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" Jun 08 15:33:32.619006 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.618989 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tpwvw" Jun 08 15:33:32.628700 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.628684 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-99v6k" Jun 08 15:33:32.635247 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.635233 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:32.890204 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.890116 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs\") pod \"network-metrics-daemon-hct6z\" (UID: \"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58\") " pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:32.890204 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.890160 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret\") pod \"global-pull-secret-syncer-r2mpb\" (UID: \"1e9d36fd-7367-43bf-8a04-3675efc9957d\") " pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:32.890417 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.890288 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 15:33:32.890417 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.890293 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 15:33:32.890417 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.890355 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret podName:1e9d36fd-7367-43bf-8a04-3675efc9957d nodeName:}" failed. No retries permitted until 2026-06-08 15:33:33.890335949 +0000 UTC m=+3.979466977 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret") pod "global-pull-secret-syncer-r2mpb" (UID: "1e9d36fd-7367-43bf-8a04-3675efc9957d") : object "kube-system"/"original-pull-secret" not registered Jun 08 15:33:32.890417 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.890374 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs podName:75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58 nodeName:}" failed. No retries permitted until 2026-06-08 15:33:33.890363995 +0000 UTC m=+3.979495023 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs") pod "network-metrics-daemon-hct6z" (UID: "75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 15:33:32.991313 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:32.991283 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5krxh\" (UniqueName: \"kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh\") pod \"network-check-target-mvb2c\" (UID: \"7711bd0d-d822-4d5d-b62f-84f831446da0\") " pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:32.991447 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.991419 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 15:33:32.991447 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.991438 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 15:33:32.991543 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.991449 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5krxh for pod openshift-network-diagnostics/network-check-target-mvb2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 15:33:32.991543 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:32.991519 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh podName:7711bd0d-d822-4d5d-b62f-84f831446da0 nodeName:}" failed. No retries permitted until 2026-06-08 15:33:33.991504868 +0000 UTC m=+4.080635899 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5krxh" (UniqueName: "kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh") pod "network-check-target-mvb2c" (UID: "7711bd0d-d822-4d5d-b62f-84f831446da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 15:33:33.039774 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:33.039744 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefff1108_d45f_4dd8_9cf8_24cbbd122b87.slice/crio-1cadb1b8fe6bc4e44e01550761a0bb68d906f5e497affb944d8abed0dde90c4b WatchSource:0}: Error finding container 1cadb1b8fe6bc4e44e01550761a0bb68d906f5e497affb944d8abed0dde90c4b: Status 404 returned error can't find the container with id 1cadb1b8fe6bc4e44e01550761a0bb68d906f5e497affb944d8abed0dde90c4b Jun 08 15:33:33.041528 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:33.041500 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b749aee_cd21_478f_92b7_544e0df2045e.slice/crio-187466d8c7d8670642a8ed7f6647f28b0907fdd222db7edb6dbb8a8c3f2f4acd WatchSource:0}: Error finding container 187466d8c7d8670642a8ed7f6647f28b0907fdd222db7edb6dbb8a8c3f2f4acd: Status 404 returned error can't find the container with id 187466d8c7d8670642a8ed7f6647f28b0907fdd222db7edb6dbb8a8c3f2f4acd Jun 08 15:33:33.044871 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:33.044850 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9865e636_5527_48fe_b4a8_de43e6b507ec.slice/crio-2ebfb054f76651cc06e5285ccfe6cf7bd61b20448f26486d60c61ea2ada62a96 WatchSource:0}: Error finding container 2ebfb054f76651cc06e5285ccfe6cf7bd61b20448f26486d60c61ea2ada62a96: Status 404 returned error can't find the container with id 2ebfb054f76651cc06e5285ccfe6cf7bd61b20448f26486d60c61ea2ada62a96 Jun 08 15:33:33.046272 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:33.046241 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd777d945_3cc7_4e04_b64b_f021f129aea7.slice/crio-25f859b81ead587ef68cd35cc8c45ff3398899fe0c85a901a7223a1253887600 WatchSource:0}: Error finding container 25f859b81ead587ef68cd35cc8c45ff3398899fe0c85a901a7223a1253887600: Status 404 returned error can't find the container with id 25f859b81ead587ef68cd35cc8c45ff3398899fe0c85a901a7223a1253887600 Jun 08 15:33:33.046950 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:33:33.046928 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7cad45c_9b89_43d6_bd57_7b3194b1d7c3.slice/crio-545735bddce39eb29c22d4adad5bca02cc8c9627447d7aa4dfcf359886bb30de WatchSource:0}: Error finding container 545735bddce39eb29c22d4adad5bca02cc8c9627447d7aa4dfcf359886bb30de: Status 404 returned error can't find the container with id 545735bddce39eb29c22d4adad5bca02cc8c9627447d7aa4dfcf359886bb30de Jun 08 15:33:33.309811 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.309650 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-07 15:28:32 +0000 UTC" deadline="2027-11-17 00:02:40.004332795 +0000 UTC" Jun 08 15:33:33.310062 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.309812 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12632h29m6.694524696s" Jun 08 15:33:33.342205 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.342172 2557 generic.go:356] "Generic (PLEG): container finished" podID="35c8ede3c284d0621d6a95a5857fea78" containerID="db7283247ae7ebb1e6b43e246c8dfd6580b11e1fd80b31218841a442b7dd66d2" exitCode=0 Jun 08 15:33:33.342316 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.342257 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal" event={"ID":"35c8ede3c284d0621d6a95a5857fea78","Type":"ContainerDied","Data":"db7283247ae7ebb1e6b43e246c8dfd6580b11e1fd80b31218841a442b7dd66d2"} Jun 08 15:33:33.343263 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.343237 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8gf2f" event={"ID":"ff5279a7-eec2-4e1b-a38f-ebee10911dd1","Type":"ContainerStarted","Data":"9a2f74dac04c3aba81ed814c201a087b0f579b3b1a9fc09dd0911c373b84bab0"} Jun 08 15:33:33.344349 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.344324 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-99v6k" event={"ID":"3be431e0-395c-47b8-a975-37cf4b23933f","Type":"ContainerStarted","Data":"7e749808427be6d55e378d5f2d6f7824a75e8eb5b631f8a25fa4076f69d52a1d"} Jun 08 15:33:33.345507 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.345489 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5t7fb" event={"ID":"d777d945-3cc7-4e04-b64b-f021f129aea7","Type":"ContainerStarted","Data":"25f859b81ead587ef68cd35cc8c45ff3398899fe0c85a901a7223a1253887600"} Jun 08 15:33:33.346434 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.346416 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l8rfj" event={"ID":"9865e636-5527-48fe-b4a8-de43e6b507ec","Type":"ContainerStarted","Data":"2ebfb054f76651cc06e5285ccfe6cf7bd61b20448f26486d60c61ea2ada62a96"} Jun 08 15:33:33.347710 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.347684 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-81.ec2.internal" event={"ID":"0d691a0e751fe5c9ef931276cec8b51d","Type":"ContainerStarted","Data":"d7042f27a5db40da6f9634e8946e3dc6f179bb986d91c61800bbc4686e5cc3e0"} Jun 08 15:33:33.348865 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.348845 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8x928" event={"ID":"1e3cc3eb-f4c6-416c-bcb8-677729009da4","Type":"ContainerStarted","Data":"5a7710c826800c828a4bf60151eaa36c99a048e85d4e15422b493d0e063d1082"} Jun 08 15:33:33.349913 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.349893 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qmt5n" event={"ID":"6a8b4c5c-e2c8-4a49-87b2-e80996d54193","Type":"ContainerStarted","Data":"55732bad2b91db9cb1700211d8331c44a57dbbe8aab6c5c55dbc7af61a450608"} Jun 08 15:33:33.350900 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.350879 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tpwvw" event={"ID":"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3","Type":"ContainerStarted","Data":"545735bddce39eb29c22d4adad5bca02cc8c9627447d7aa4dfcf359886bb30de"} Jun 08 15:33:33.351967 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.351947 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" event={"ID":"9b749aee-cd21-478f-92b7-544e0df2045e","Type":"ContainerStarted","Data":"187466d8c7d8670642a8ed7f6647f28b0907fdd222db7edb6dbb8a8c3f2f4acd"} Jun 08 15:33:33.352882 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.352863 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" event={"ID":"efff1108-d45f-4dd8-9cf8-24cbbd122b87","Type":"ContainerStarted","Data":"1cadb1b8fe6bc4e44e01550761a0bb68d906f5e497affb944d8abed0dde90c4b"} Jun 08 15:33:33.897823 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.897787 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs\") pod \"network-metrics-daemon-hct6z\" (UID: \"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58\") " pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:33.897989 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.897838 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret\") pod \"global-pull-secret-syncer-r2mpb\" (UID: \"1e9d36fd-7367-43bf-8a04-3675efc9957d\") " pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:33.898068 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:33.898022 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 15:33:33.898119 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:33.898085 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret podName:1e9d36fd-7367-43bf-8a04-3675efc9957d nodeName:}" failed. No retries permitted until 2026-06-08 15:33:35.898067081 +0000 UTC m=+5.987198112 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret") pod "global-pull-secret-syncer-r2mpb" (UID: "1e9d36fd-7367-43bf-8a04-3675efc9957d") : object "kube-system"/"original-pull-secret" not registered Jun 08 15:33:33.898513 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:33.898490 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 15:33:33.898595 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:33.898539 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs podName:75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58 nodeName:}" failed. No retries permitted until 2026-06-08 15:33:35.898524793 +0000 UTC m=+5.987655835 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs") pod "network-metrics-daemon-hct6z" (UID: "75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 15:33:33.998279 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:33.998244 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5krxh\" (UniqueName: \"kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh\") pod \"network-check-target-mvb2c\" (UID: \"7711bd0d-d822-4d5d-b62f-84f831446da0\") " pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:33.998482 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:33.998436 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 15:33:33.998482 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:33.998475 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 15:33:33.998603 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:33.998489 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5krxh for pod openshift-network-diagnostics/network-check-target-mvb2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 15:33:33.998603 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:33.998544 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh podName:7711bd0d-d822-4d5d-b62f-84f831446da0 nodeName:}" failed. No retries permitted until 2026-06-08 15:33:35.998526438 +0000 UTC m=+6.087657485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5krxh" (UniqueName: "kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh") pod "network-check-target-mvb2c" (UID: "7711bd0d-d822-4d5d-b62f-84f831446da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 15:33:34.310209 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:34.310090 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-07 15:28:32 +0000 UTC" deadline="2028-03-16 13:31:15.722647408 +0000 UTC" Jun 08 15:33:34.310209 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:34.310127 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15525h57m41.412525224s" Jun 08 15:33:34.335081 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:34.335053 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:34.335232 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:34.335182 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:33:34.336204 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:34.335477 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:34.336204 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:34.335490 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:34.336204 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:34.335611 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:33:34.336204 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:34.336171 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:33:34.382831 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:34.382736 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal" event={"ID":"35c8ede3c284d0621d6a95a5857fea78","Type":"ContainerStarted","Data":"2879483ba51c293965bb5c7b26d9476ba24d69048a5173012dbde55d06612e8f"} Jun 08 15:33:34.397487 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:34.396554 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-81.ec2.internal" podStartSLOduration=3.396537195 podStartE2EDuration="3.396537195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 15:33:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:33:33.368332634 +0000 UTC m=+3.457463683" watchObservedRunningTime="2026-06-08 15:33:34.396537195 +0000 UTC m=+4.485668246" Jun 08 15:33:35.917312 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:35.916528 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs\") pod \"network-metrics-daemon-hct6z\" (UID: \"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58\") " pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:35.917312 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:35.916578 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret\") pod \"global-pull-secret-syncer-r2mpb\" (UID: \"1e9d36fd-7367-43bf-8a04-3675efc9957d\") " pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:35.917312 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:35.916749 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 15:33:35.917312 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:35.916811 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret podName:1e9d36fd-7367-43bf-8a04-3675efc9957d nodeName:}" failed. No retries permitted until 2026-06-08 15:33:39.916792027 +0000 UTC m=+10.005923058 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret") pod "global-pull-secret-syncer-r2mpb" (UID: "1e9d36fd-7367-43bf-8a04-3675efc9957d") : object "kube-system"/"original-pull-secret" not registered Jun 08 15:33:35.917312 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:35.917229 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 15:33:35.917312 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:35.917276 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs podName:75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58 nodeName:}" failed. No retries permitted until 2026-06-08 15:33:39.917261846 +0000 UTC m=+10.006392880 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs") pod "network-metrics-daemon-hct6z" (UID: "75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 15:33:36.017807 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:36.017683 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5krxh\" (UniqueName: \"kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh\") pod \"network-check-target-mvb2c\" (UID: \"7711bd0d-d822-4d5d-b62f-84f831446da0\") " pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:36.018032 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:36.017861 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 15:33:36.018032 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:36.017893 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 15:33:36.018032 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:36.017908 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5krxh for pod openshift-network-diagnostics/network-check-target-mvb2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 15:33:36.018032 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:36.017970 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh podName:7711bd0d-d822-4d5d-b62f-84f831446da0 nodeName:}" failed. No retries permitted until 2026-06-08 15:33:40.017950005 +0000 UTC m=+10.107081036 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5krxh" (UniqueName: "kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh") pod "network-check-target-mvb2c" (UID: "7711bd0d-d822-4d5d-b62f-84f831446da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 15:33:36.333575 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:36.333502 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:36.333732 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:36.333640 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:33:36.334188 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:36.334000 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:36.334188 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:36.334088 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:33:36.334188 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:36.334144 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:36.334383 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:36.334227 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:33:38.331966 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:38.331936 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:38.332343 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:38.331941 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:38.332343 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:38.332081 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:38.332343 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:38.332096 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:33:38.332343 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:38.332210 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:33:38.332343 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:38.332289 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:33:39.949945 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:39.949797 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs\") pod \"network-metrics-daemon-hct6z\" (UID: \"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58\") " pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:39.949945 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:39.949873 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret\") pod \"global-pull-secret-syncer-r2mpb\" (UID: \"1e9d36fd-7367-43bf-8a04-3675efc9957d\") " pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:39.950608 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:39.949958 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 15:33:39.950608 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:39.950020 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 15:33:39.950608 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:39.950024 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs podName:75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58 nodeName:}" failed. No retries permitted until 2026-06-08 15:33:47.950004649 +0000 UTC m=+18.039135688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs") pod "network-metrics-daemon-hct6z" (UID: "75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 15:33:39.950608 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:39.950093 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret podName:1e9d36fd-7367-43bf-8a04-3675efc9957d nodeName:}" failed. No retries permitted until 2026-06-08 15:33:47.950075465 +0000 UTC m=+18.039206499 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret") pod "global-pull-secret-syncer-r2mpb" (UID: "1e9d36fd-7367-43bf-8a04-3675efc9957d") : object "kube-system"/"original-pull-secret" not registered Jun 08 15:33:40.051089 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:40.051052 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5krxh\" (UniqueName: \"kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh\") pod \"network-check-target-mvb2c\" (UID: \"7711bd0d-d822-4d5d-b62f-84f831446da0\") " pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:40.051265 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:40.051248 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 15:33:40.051334 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:40.051278 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 15:33:40.051334 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:40.051292 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5krxh for pod openshift-network-diagnostics/network-check-target-mvb2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 15:33:40.051447 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:40.051355 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh podName:7711bd0d-d822-4d5d-b62f-84f831446da0 nodeName:}" failed. No retries permitted until 2026-06-08 15:33:48.051335927 +0000 UTC m=+18.140466957 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5krxh" (UniqueName: "kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh") pod "network-check-target-mvb2c" (UID: "7711bd0d-d822-4d5d-b62f-84f831446da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 15:33:40.333321 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:40.333246 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:40.333321 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:40.333269 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:40.333539 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:40.333370 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:33:40.333539 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:40.333432 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:33:40.333539 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:40.333502 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:40.333842 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:40.333566 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:33:42.332824 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:42.332791 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:42.333294 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:42.332791 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:42.333294 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:42.332908 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:33:42.333294 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:42.332791 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:42.333294 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:42.333016 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:33:42.333533 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:42.333216 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:33:44.332817 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:44.332785 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:44.333307 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:44.332785 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:44.333307 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:44.332903 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:33:44.333307 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:44.333011 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:33:44.333307 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:44.332790 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:44.333307 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:44.333114 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:33:46.332173 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:46.332141 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:46.332732 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:46.332141 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:46.332732 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:46.332273 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:33:46.332732 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:46.332370 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:46.332732 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:46.332358 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:33:46.332732 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:46.332447 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:33:48.008191 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:48.008155 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs\") pod \"network-metrics-daemon-hct6z\" (UID: \"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58\") " pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:48.008191 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:48.008197 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret\") pod \"global-pull-secret-syncer-r2mpb\" (UID: \"1e9d36fd-7367-43bf-8a04-3675efc9957d\") " pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:48.008754 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:48.008311 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 15:33:48.008754 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:48.008390 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs podName:75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58 nodeName:}" failed. No retries permitted until 2026-06-08 15:34:04.008367304 +0000 UTC m=+34.097498362 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs") pod "network-metrics-daemon-hct6z" (UID: "75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 15:33:48.008754 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:48.008318 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 15:33:48.008754 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:48.008457 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret podName:1e9d36fd-7367-43bf-8a04-3675efc9957d nodeName:}" failed. No retries permitted until 2026-06-08 15:34:04.008443555 +0000 UTC m=+34.097574583 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret") pod "global-pull-secret-syncer-r2mpb" (UID: "1e9d36fd-7367-43bf-8a04-3675efc9957d") : object "kube-system"/"original-pull-secret" not registered Jun 08 15:33:48.108811 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:48.108775 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5krxh\" (UniqueName: \"kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh\") pod \"network-check-target-mvb2c\" (UID: \"7711bd0d-d822-4d5d-b62f-84f831446da0\") " pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:48.108981 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:48.108914 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 15:33:48.108981 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:48.108929 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 15:33:48.108981 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:48.108939 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5krxh for pod openshift-network-diagnostics/network-check-target-mvb2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 15:33:48.109125 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:48.108995 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh podName:7711bd0d-d822-4d5d-b62f-84f831446da0 nodeName:}" failed. No retries permitted until 2026-06-08 15:34:04.108978971 +0000 UTC m=+34.198109999 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5krxh" (UniqueName: "kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh") pod "network-check-target-mvb2c" (UID: "7711bd0d-d822-4d5d-b62f-84f831446da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 15:33:48.332842 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:48.332771 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:48.332999 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:48.332775 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:48.332999 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:48.332900 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:33:48.332999 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:48.332911 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:48.333126 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:48.333085 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:33:48.333126 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:48.333106 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:33:50.333505 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:50.333416 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:50.333915 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:50.333529 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:50.333915 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:50.333549 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:33:50.333915 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:50.333622 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:33:50.333915 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:50.333677 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:50.333915 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:50.333786 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:33:52.332937 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.332562 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:52.333621 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.332608 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:52.333621 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:52.333046 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:33:52.333621 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:52.333106 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:33:52.333621 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.332718 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:52.333621 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:52.333195 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:33:52.418561 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.418521 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qmt5n" event={"ID":"6a8b4c5c-e2c8-4a49-87b2-e80996d54193","Type":"ContainerStarted","Data":"0fb3cec4f3713c21f7ea9e32173db3c6a4154d8c01c1a637c3dfbde46af3e2a3"} Jun 08 15:33:52.420294 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.420264 2557 generic.go:356] "Generic (PLEG): container finished" podID="a7cad45c-9b89-43d6-bd57-7b3194b1d7c3" containerID="8e26d109e826490bd01fdc889ef2697bbebfc1f694d5ec57dd614a7febe528c9" exitCode=0 Jun 08 15:33:52.420416 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.420348 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tpwvw" event={"ID":"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3","Type":"ContainerDied","Data":"8e26d109e826490bd01fdc889ef2697bbebfc1f694d5ec57dd614a7febe528c9"} Jun 08 15:33:52.422337 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.422313 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" event={"ID":"9b749aee-cd21-478f-92b7-544e0df2045e","Type":"ContainerStarted","Data":"fc00543e1790973ee37a39d0231c4211321eb433d22f1aa60be5e51dc54cde92"} Jun 08 15:33:52.425657 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.425636 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 15:33:52.426127 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.426106 2557 generic.go:356] "Generic (PLEG): container finished" podID="efff1108-d45f-4dd8-9cf8-24cbbd122b87" containerID="6ce3074e5f885c2429570c27244db3ab2f774114adf97d0e5bd4e7bddcd9dab8" exitCode=1 Jun 08 15:33:52.426225 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.426152 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" event={"ID":"efff1108-d45f-4dd8-9cf8-24cbbd122b87","Type":"ContainerStarted","Data":"9617329ac7d4ac3dcc51427ae3cb404486c881c050953d0b475377bbe039da9e"} Jun 08 15:33:52.426296 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.426238 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" event={"ID":"efff1108-d45f-4dd8-9cf8-24cbbd122b87","Type":"ContainerStarted","Data":"59c4a555efd094c3ad1c080b15be0081dacd3c6e1be5a4490ca2d0ce924be45e"} Jun 08 15:33:52.426296 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.426292 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" event={"ID":"efff1108-d45f-4dd8-9cf8-24cbbd122b87","Type":"ContainerStarted","Data":"96012071c512c4c5e5a9f19dcc6808f6cc2727fcdc975f7ba242bf7141fff19d"} Jun 08 15:33:52.426368 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.426341 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" event={"ID":"efff1108-d45f-4dd8-9cf8-24cbbd122b87","Type":"ContainerStarted","Data":"3ab69c2c1be3ad8e543064ae8740e0612eb580b49ac23efd24d98f6ce119d730"} Jun 08 15:33:52.426418 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.426386 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" event={"ID":"efff1108-d45f-4dd8-9cf8-24cbbd122b87","Type":"ContainerDied","Data":"6ce3074e5f885c2429570c27244db3ab2f774114adf97d0e5bd4e7bddcd9dab8"} Jun 08 15:33:52.426489 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.426434 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" event={"ID":"efff1108-d45f-4dd8-9cf8-24cbbd122b87","Type":"ContainerStarted","Data":"6fb024715385b7d373cee9819d31e7a972c5c25b4d5973a2b951d5c30f1628c9"} Jun 08 15:33:52.427833 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.427811 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8gf2f" event={"ID":"ff5279a7-eec2-4e1b-a38f-ebee10911dd1","Type":"ContainerStarted","Data":"487ecb6ce631ecb516e650e94dc5e30acaa8ae2a16a54c11f56cc2b268df3ac6"} Jun 08 15:33:52.429679 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.429656 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5t7fb" event={"ID":"d777d945-3cc7-4e04-b64b-f021f129aea7","Type":"ContainerStarted","Data":"d395501bd1e4ca301e5eda31550c233afb602a6bf61046cacb1fcd67988bea04"} Jun 08 15:33:52.431166 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.431121 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l8rfj" event={"ID":"9865e636-5527-48fe-b4a8-de43e6b507ec","Type":"ContainerStarted","Data":"2b8b06b3053353899c552fa921c450b8842fe762e2937dc10bd072100c30afa0"} Jun 08 15:33:52.432912 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.432892 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8x928" event={"ID":"1e3cc3eb-f4c6-416c-bcb8-677729009da4","Type":"ContainerStarted","Data":"0d7b2d21a7ab73af824f9305eabee6e06a7a72764bef4b3913efab26b100d1fd"} Jun 08 15:33:52.437231 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.437185 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qmt5n" podStartSLOduration=3.78347222 podStartE2EDuration="22.437173639s" totalImagesPullingTime="18.653701419s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 15:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:33:52.436568236 +0000 UTC m=+22.525699287" watchObservedRunningTime="2026-06-08 15:33:52.437173639 +0000 UTC m=+22.526304708" Jun 08 15:33:52.437551 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.437517 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-81.ec2.internal" podStartSLOduration=21.437504964 podStartE2EDuration="21.437504964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 15:33:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:33:34.397795447 +0000 UTC m=+4.486926498" watchObservedRunningTime="2026-06-08 15:33:52.437504964 +0000 UTC m=+22.526636010" Jun 08 15:33:52.449229 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.449193 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l8rfj" podStartSLOduration=3.8051407470000003 podStartE2EDuration="22.449186245s" totalImagesPullingTime="18.644045498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 15:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:33:52.448872152 +0000 UTC m=+22.538003203" watchObservedRunningTime="2026-06-08 15:33:52.449186245 +0000 UTC m=+22.538317295" Jun 08 15:33:52.462325 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.462275 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5t7fb" podStartSLOduration=3.822868334 podStartE2EDuration="22.462258453s" totalImagesPullingTime="18.639390119s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 15:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:33:52.462224887 +0000 UTC m=+22.551355938" watchObservedRunningTime="2026-06-08 15:33:52.462258453 +0000 UTC m=+22.551389506" Jun 08 15:33:52.475652 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.475614 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8gf2f" podStartSLOduration=12.246787474 podStartE2EDuration="22.47560141s" totalImagesPullingTime="10.228813936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:33:52.475285729 +0000 UTC m=+22.564416780" watchObservedRunningTime="2026-06-08 15:33:52.47560141 +0000 UTC m=+22.564732460" Jun 08 15:33:52.514576 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:52.514531 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8x928" podStartSLOduration=3.860956176 podStartE2EDuration="22.514512018s" totalImagesPullingTime="18.653555842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 15:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:33:52.514204686 +0000 UTC m=+22.603335746" watchObservedRunningTime="2026-06-08 15:33:52.514512018 +0000 UTC m=+22.603643069" Jun 08 15:33:53.334904 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:53.334859 2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 08 15:33:53.437074 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:53.437038 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-99v6k" event={"ID":"3be431e0-395c-47b8-a975-37cf4b23933f","Type":"ContainerStarted","Data":"907f7b956fecd7b45165df4af7143d49ed3284eb93493d9f8113b6ea399238cd"} Jun 08 15:33:53.439123 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:53.439094 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" event={"ID":"9b749aee-cd21-478f-92b7-544e0df2045e","Type":"ContainerStarted","Data":"d6c7c40bd8b7df83c136e872707972eb5820387cca21e5d2aa21c3a73799c9d5"} Jun 08 15:33:53.450162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:53.450089 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-99v6k" podStartSLOduration=4.810890794 podStartE2EDuration="23.450075477s" totalImagesPullingTime="18.639184683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 15:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:33:53.449889521 +0000 UTC m=+23.539020571" watchObservedRunningTime="2026-06-08 15:33:53.450075477 +0000 UTC m=+23.539206521" Jun 08 15:33:54.076721 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:54.076685 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8gf2f" Jun 08 15:33:54.077215 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:54.077193 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8gf2f" Jun 08 15:33:54.332508 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:54.332404 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:54.332508 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:54.332418 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:54.332728 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:54.332553 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:54.332728 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:54.332566 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:33:54.332728 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:54.332653 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:33:54.332880 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:54.332760 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:33:54.334925 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:54.334840 2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-08T15:33:53.334899624Z","UUID":"6c534804-6e03-44ff-add5-d7f250f90be4","Handler":null,"Name":"","Endpoint":""} Jun 08 15:33:54.338055 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:54.338035 2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 08 15:33:54.338168 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:54.338077 2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 08 15:33:54.444013 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:54.443980 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" event={"ID":"9b749aee-cd21-478f-92b7-544e0df2045e","Type":"ContainerStarted","Data":"2572e8cb591f723a33b9da423d203baf223cf2c87278d7387c0855cf89ce4081"} Jun 08 15:33:54.444282 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:54.444260 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8gf2f" Jun 08 15:33:54.444711 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:54.444695 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8gf2f" Jun 08 15:33:54.460119 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:54.460079 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hsnm6" podStartSLOduration=3.767679297 podStartE2EDuration="24.460066597s" totalImagesPullingTime="20.6923873s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 15:33:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:33:54.459383287 +0000 UTC m=+24.548514349" watchObservedRunningTime="2026-06-08 15:33:54.460066597 +0000 UTC m=+24.549197644" Jun 08 15:33:55.449249 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:55.449224 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 15:33:55.449645 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:55.449617 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" event={"ID":"efff1108-d45f-4dd8-9cf8-24cbbd122b87","Type":"ContainerStarted","Data":"7caad72de937522ad78ea173a23433c218a116f6df9a6bf07e647fb3fb6ac989"} Jun 08 15:33:56.332423 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:56.332225 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:56.332623 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:56.332234 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:56.332623 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:56.332540 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:33:56.332623 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:56.332605 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:33:56.332798 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:56.332234 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:56.332798 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:56.332709 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:33:57.456734 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:57.456715 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 15:33:57.457326 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:57.457043 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" event={"ID":"efff1108-d45f-4dd8-9cf8-24cbbd122b87","Type":"ContainerStarted","Data":"891d485c002a7fd338722bc467bf043373c848f7d75fb1e115b982682c402001"} Jun 08 15:33:57.457391 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:57.457335 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:57.457434 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:57.457410 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:57.457574 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:57.457561 2557 scope.go:117] "RemoveContainer" containerID="6ce3074e5f885c2429570c27244db3ab2f774114adf97d0e5bd4e7bddcd9dab8" Jun 08 15:33:57.471733 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:57.471714 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:58.332312 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:58.332224 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:58.332312 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:58.332266 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:58.332312 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:58.332227 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:58.332588 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:58.332353 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:33:58.332588 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:58.332451 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:33:58.332588 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:58.332551 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:33:58.461476 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:58.461437 2557 generic.go:356] "Generic (PLEG): container finished" podID="a7cad45c-9b89-43d6-bd57-7b3194b1d7c3" containerID="8553d4817adf4f80f2eae56b7d8683c33eecaef18a57e0956b64dfa942e92cba" exitCode=0 Jun 08 15:33:58.461866 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:58.461526 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tpwvw" event={"ID":"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3","Type":"ContainerDied","Data":"8553d4817adf4f80f2eae56b7d8683c33eecaef18a57e0956b64dfa942e92cba"} Jun 08 15:33:58.465504 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:58.465363 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 15:33:58.465795 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:58.465772 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" event={"ID":"efff1108-d45f-4dd8-9cf8-24cbbd122b87","Type":"ContainerStarted","Data":"b27f0ce390c26d46381b278e0944bc6be11efa04dc9486f467cfa3294accc883"} Jun 08 15:33:58.465963 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:58.465946 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:58.480672 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:58.480654 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:33:58.504978 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:58.504935 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" podStartSLOduration=9.774033056 podStartE2EDuration="28.504920816s" totalImagesPullingTime="18.73088776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 15:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:33:58.504354088 +0000 UTC m=+28.593485138" watchObservedRunningTime="2026-06-08 15:33:58.504920816 +0000 UTC m=+28.594051866" Jun 08 15:33:59.312448 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:59.312415 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mvb2c"] Jun 08 15:33:59.312669 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:59.312540 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:33:59.312722 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:59.312658 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:33:59.316902 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:59.316864 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-r2mpb"] Jun 08 15:33:59.317048 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:59.316970 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:33:59.317106 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:59.317068 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:33:59.317484 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:59.317448 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hct6z"] Jun 08 15:33:59.317582 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:33:59.317550 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:33:59.317668 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:33:59.317649 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:34:00.471031 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:00.470999 2557 generic.go:356] "Generic (PLEG): container finished" podID="a7cad45c-9b89-43d6-bd57-7b3194b1d7c3" containerID="97a72d226a4392833549d19922764be363c8fc28723b2f8b877b6f7db70fe0cb" exitCode=0 Jun 08 15:34:00.471402 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:00.471077 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tpwvw" event={"ID":"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3","Type":"ContainerDied","Data":"97a72d226a4392833549d19922764be363c8fc28723b2f8b877b6f7db70fe0cb"} Jun 08 15:34:01.332852 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:01.332829 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:34:01.332917 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:01.332848 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:34:01.332917 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:01.332856 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:34:01.332988 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:01.332955 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:34:01.333090 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:01.333073 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:34:01.333167 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:01.333152 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:34:01.475630 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:01.475599 2557 generic.go:356] "Generic (PLEG): container finished" podID="a7cad45c-9b89-43d6-bd57-7b3194b1d7c3" containerID="1933d74bdc1957d1ac245d7e4b26ffdf778748bb1879441b45b7a6301765cd04" exitCode=0 Jun 08 15:34:01.475949 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:01.475661 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tpwvw" event={"ID":"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3","Type":"ContainerDied","Data":"1933d74bdc1957d1ac245d7e4b26ffdf778748bb1879441b45b7a6301765cd04"} Jun 08 15:34:03.332559 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:03.332480 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:34:03.332559 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:03.332510 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:34:03.332559 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:03.332514 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:34:03.333011 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:03.332606 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:34:03.333011 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:03.332682 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r2mpb" podUID="1e9d36fd-7367-43bf-8a04-3675efc9957d" Jun 08 15:34:03.333011 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:03.332763 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mvb2c" podUID="7711bd0d-d822-4d5d-b62f-84f831446da0" Jun 08 15:34:04.023071 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.023041 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs\") pod \"network-metrics-daemon-hct6z\" (UID: \"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58\") " pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:34:04.023240 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.023075 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret\") pod \"global-pull-secret-syncer-r2mpb\" (UID: \"1e9d36fd-7367-43bf-8a04-3675efc9957d\") " pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:34:04.023240 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:04.023206 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 15:34:04.023240 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:04.023217 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 15:34:04.023372 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:04.023273 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret podName:1e9d36fd-7367-43bf-8a04-3675efc9957d nodeName:}" failed. No retries permitted until 2026-06-08 15:34:36.023259596 +0000 UTC m=+66.112390624 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret") pod "global-pull-secret-syncer-r2mpb" (UID: "1e9d36fd-7367-43bf-8a04-3675efc9957d") : object "kube-system"/"original-pull-secret" not registered Jun 08 15:34:04.023372 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:04.023288 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs podName:75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58 nodeName:}" failed. No retries permitted until 2026-06-08 15:34:36.023281187 +0000 UTC m=+66.112412228 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs") pod "network-metrics-daemon-hct6z" (UID: "75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 15:34:04.123579 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.123548 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5krxh\" (UniqueName: \"kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh\") pod \"network-check-target-mvb2c\" (UID: \"7711bd0d-d822-4d5d-b62f-84f831446da0\") " pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:34:04.123811 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:04.123720 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 15:34:04.123811 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:04.123739 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 15:34:04.123811 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:04.123751 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5krxh for pod openshift-network-diagnostics/network-check-target-mvb2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 15:34:04.123931 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:04.123812 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh podName:7711bd0d-d822-4d5d-b62f-84f831446da0 nodeName:}" failed. No retries permitted until 2026-06-08 15:34:36.12379839 +0000 UTC m=+66.212929418 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5krxh" (UniqueName: "kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh") pod "network-check-target-mvb2c" (UID: "7711bd0d-d822-4d5d-b62f-84f831446da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 15:34:04.702525 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.702278 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-81.ec2.internal" event="NodeReady" Jun 08 15:34:04.702910 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.702653 2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 08 15:34:04.735707 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.735676 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84f8f6f6b6-jx5jx"] Jun 08 15:34:04.759393 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.759368 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84f8f6f6b6-jx5jx"] Jun 08 15:34:04.759393 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.759403 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mvlgf"] Jun 08 15:34:04.759610 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.759518 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.762092 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.762060 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 08 15:34:04.763911 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.762282 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 08 15:34:04.763911 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.762590 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mdbb6\"" Jun 08 15:34:04.763911 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.762937 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 08 15:34:04.770820 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.770797 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 08 15:34:04.773346 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.773325 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-t8nc2"] Jun 08 15:34:04.773505 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.773453 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mvlgf" Jun 08 15:34:04.775516 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.775495 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 08 15:34:04.775687 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.775671 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 08 15:34:04.775787 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.775750 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nvn49\"" Jun 08 15:34:04.775848 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.775810 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 08 15:34:04.789572 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.789537 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mvlgf"] Jun 08 15:34:04.789572 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.789568 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t8nc2"] Jun 08 15:34:04.789729 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.789662 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:04.791946 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.791929 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p7nwf\"" Jun 08 15:34:04.792045 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.791932 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 08 15:34:04.792216 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.792192 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 08 15:34:04.830115 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.830085 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a0a2ada9-1340-40f3-b339-f2a682aecf8d-installation-pull-secrets\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.830344 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.830120 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-certificates\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.830344 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.830139 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-bound-sa-token\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.830344 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.830222 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.830344 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.830275 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a0a2ada9-1340-40f3-b339-f2a682aecf8d-ca-trust-extracted\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.830344 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.830297 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0a2ada9-1340-40f3-b339-f2a682aecf8d-trusted-ca\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.830344 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.830329 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7mzj\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-kube-api-access-r7mzj\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.830612 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.830392 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a0a2ada9-1340-40f3-b339-f2a682aecf8d-image-registry-private-configuration\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.898540 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.898502 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:04.931392 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.931365 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn4rt\" (UniqueName: \"kubernetes.io/projected/ac6d6aa0-340a-4156-aa2d-60444b6f4fcb-kube-api-access-sn4rt\") pod \"ingress-canary-mvlgf\" (UID: \"ac6d6aa0-340a-4156-aa2d-60444b6f4fcb\") " pod="openshift-ingress-canary/ingress-canary-mvlgf" Jun 08 15:34:04.931590 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.931412 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a0a2ada9-1340-40f3-b339-f2a682aecf8d-ca-trust-extracted\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.931590 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.931458 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0a2ada9-1340-40f3-b339-f2a682aecf8d-trusted-ca\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.931590 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.931506 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac6d6aa0-340a-4156-aa2d-60444b6f4fcb-cert\") pod \"ingress-canary-mvlgf\" (UID: \"ac6d6aa0-340a-4156-aa2d-60444b6f4fcb\") " pod="openshift-ingress-canary/ingress-canary-mvlgf" Jun 08 15:34:04.931590 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.931553 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7mzj\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-kube-api-access-r7mzj\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.931790 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.931595 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a0a2ada9-1340-40f3-b339-f2a682aecf8d-image-registry-private-configuration\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.931790 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.931629 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9492b29-8d64-43ca-9f70-250f32707cfd-tmp-dir\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:04.931790 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.931663 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b9492b29-8d64-43ca-9f70-250f32707cfd-config-volume\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:04.931790 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.931687 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:04.931790 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.931722 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a0a2ada9-1340-40f3-b339-f2a682aecf8d-installation-pull-secrets\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.931790 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.931756 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-certificates\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.931790 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.931788 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-bound-sa-token\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.932127 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.931824 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a0a2ada9-1340-40f3-b339-f2a682aecf8d-ca-trust-extracted\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.932127 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.931835 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.932127 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.931861 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx6cf\" (UniqueName: \"kubernetes.io/projected/b9492b29-8d64-43ca-9f70-250f32707cfd-kube-api-access-tx6cf\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:04.932127 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:04.932067 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 15:34:04.932127 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:04.932086 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f8f6f6b6-jx5jx: secret "image-registry-tls" not found Jun 08 15:34:04.932334 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:04.932143 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls podName:a0a2ada9-1340-40f3-b339-f2a682aecf8d nodeName:}" failed. No retries permitted until 2026-06-08 15:34:05.432124164 +0000 UTC m=+35.521255201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls") pod "image-registry-84f8f6f6b6-jx5jx" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d") : secret "image-registry-tls" not found Jun 08 15:34:04.932587 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.932556 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0a2ada9-1340-40f3-b339-f2a682aecf8d-trusted-ca\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.932795 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.932779 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-certificates\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.936190 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.936170 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a0a2ada9-1340-40f3-b339-f2a682aecf8d-installation-pull-secrets\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.936292 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.936185 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a0a2ada9-1340-40f3-b339-f2a682aecf8d-image-registry-private-configuration\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.939959 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.939936 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-bound-sa-token\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.940225 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.940206 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7mzj\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-kube-api-access-r7mzj\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:04.975879 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:04.975802 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:05.032685 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.032659 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx6cf\" (UniqueName: \"kubernetes.io/projected/b9492b29-8d64-43ca-9f70-250f32707cfd-kube-api-access-tx6cf\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:05.032822 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.032694 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn4rt\" (UniqueName: \"kubernetes.io/projected/ac6d6aa0-340a-4156-aa2d-60444b6f4fcb-kube-api-access-sn4rt\") pod \"ingress-canary-mvlgf\" (UID: \"ac6d6aa0-340a-4156-aa2d-60444b6f4fcb\") " pod="openshift-ingress-canary/ingress-canary-mvlgf" Jun 08 15:34:05.032822 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.032734 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac6d6aa0-340a-4156-aa2d-60444b6f4fcb-cert\") pod \"ingress-canary-mvlgf\" (UID: \"ac6d6aa0-340a-4156-aa2d-60444b6f4fcb\") " pod="openshift-ingress-canary/ingress-canary-mvlgf" Jun 08 15:34:05.032822 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.032790 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9492b29-8d64-43ca-9f70-250f32707cfd-tmp-dir\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:05.032971 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.032821 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b9492b29-8d64-43ca-9f70-250f32707cfd-config-volume\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:05.032971 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.032841 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:05.032971 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:05.032936 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 15:34:05.033184 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:05.032989 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls podName:b9492b29-8d64-43ca-9f70-250f32707cfd nodeName:}" failed. No retries permitted until 2026-06-08 15:34:05.532971732 +0000 UTC m=+35.622102759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls") pod "dns-default-t8nc2" (UID: "b9492b29-8d64-43ca-9f70-250f32707cfd") : secret "dns-default-metrics-tls" not found Jun 08 15:34:05.033252 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.033228 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9492b29-8d64-43ca-9f70-250f32707cfd-tmp-dir\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:05.033435 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.033406 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b9492b29-8d64-43ca-9f70-250f32707cfd-config-volume\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:05.035529 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.035505 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac6d6aa0-340a-4156-aa2d-60444b6f4fcb-cert\") pod \"ingress-canary-mvlgf\" (UID: \"ac6d6aa0-340a-4156-aa2d-60444b6f4fcb\") " pod="openshift-ingress-canary/ingress-canary-mvlgf" Jun 08 15:34:05.041399 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.041352 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx6cf\" (UniqueName: \"kubernetes.io/projected/b9492b29-8d64-43ca-9f70-250f32707cfd-kube-api-access-tx6cf\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:05.041994 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.041976 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn4rt\" (UniqueName: \"kubernetes.io/projected/ac6d6aa0-340a-4156-aa2d-60444b6f4fcb-kube-api-access-sn4rt\") pod \"ingress-canary-mvlgf\" (UID: \"ac6d6aa0-340a-4156-aa2d-60444b6f4fcb\") " pod="openshift-ingress-canary/ingress-canary-mvlgf" Jun 08 15:34:05.082989 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.082961 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mvlgf" Jun 08 15:34:05.332390 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.332306 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:34:05.332390 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.332327 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:34:05.332618 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.332306 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:34:05.334810 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.334785 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 08 15:34:05.334932 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.334889 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ld5bv\"" Jun 08 15:34:05.334990 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.334925 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 08 15:34:05.334990 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.334944 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-54ltp\"" Jun 08 15:34:05.334990 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.334931 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 08 15:34:05.335131 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.334893 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 08 15:34:05.435980 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.435945 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:05.436141 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:05.436096 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 15:34:05.436141 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:05.436111 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f8f6f6b6-jx5jx: secret "image-registry-tls" not found Jun 08 15:34:05.436242 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:05.436174 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls podName:a0a2ada9-1340-40f3-b339-f2a682aecf8d nodeName:}" failed. No retries permitted until 2026-06-08 15:34:06.436155726 +0000 UTC m=+36.525286757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls") pod "image-registry-84f8f6f6b6-jx5jx" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d") : secret "image-registry-tls" not found Jun 08 15:34:05.537214 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.537188 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:05.537346 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:05.537331 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 15:34:05.537412 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:05.537403 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls podName:b9492b29-8d64-43ca-9f70-250f32707cfd nodeName:}" failed. No retries permitted until 2026-06-08 15:34:06.537385439 +0000 UTC m=+36.626516468 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls") pod "dns-default-t8nc2" (UID: "b9492b29-8d64-43ca-9f70-250f32707cfd") : secret "dns-default-metrics-tls" not found Jun 08 15:34:05.664297 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:05.664254 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mvlgf"] Jun 08 15:34:05.736524 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:34:05.736493 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6d6aa0_340a_4156_aa2d_60444b6f4fcb.slice/crio-c761d2be70dd4ad9b3fbabe3375a18a4ccffa1d44436a433f32eed57c429ff66 WatchSource:0}: Error finding container c761d2be70dd4ad9b3fbabe3375a18a4ccffa1d44436a433f32eed57c429ff66: Status 404 returned error can't find the container with id c761d2be70dd4ad9b3fbabe3375a18a4ccffa1d44436a433f32eed57c429ff66 Jun 08 15:34:06.447089 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:06.446909 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:06.447089 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:06.447089 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 15:34:06.447292 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:06.447107 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f8f6f6b6-jx5jx: secret "image-registry-tls" not found Jun 08 15:34:06.447292 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:06.447168 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls podName:a0a2ada9-1340-40f3-b339-f2a682aecf8d nodeName:}" failed. No retries permitted until 2026-06-08 15:34:08.44715295 +0000 UTC m=+38.536283979 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls") pod "image-registry-84f8f6f6b6-jx5jx" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d") : secret "image-registry-tls" not found Jun 08 15:34:06.488986 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:06.488959 2557 generic.go:356] "Generic (PLEG): container finished" podID="a7cad45c-9b89-43d6-bd57-7b3194b1d7c3" containerID="c483501bce5d4226e80289cefce1383256e640bfc95ebe5afa3dbcc9b790a387" exitCode=0 Jun 08 15:34:06.489131 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:06.489043 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tpwvw" event={"ID":"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3","Type":"ContainerDied","Data":"c483501bce5d4226e80289cefce1383256e640bfc95ebe5afa3dbcc9b790a387"} Jun 08 15:34:06.490160 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:06.490137 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mvlgf" event={"ID":"ac6d6aa0-340a-4156-aa2d-60444b6f4fcb","Type":"ContainerStarted","Data":"c761d2be70dd4ad9b3fbabe3375a18a4ccffa1d44436a433f32eed57c429ff66"} Jun 08 15:34:06.548393 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:06.548361 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:06.548522 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:06.548504 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 15:34:06.548584 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:06.548575 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls podName:b9492b29-8d64-43ca-9f70-250f32707cfd nodeName:}" failed. No retries permitted until 2026-06-08 15:34:08.548551202 +0000 UTC m=+38.637682231 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls") pod "dns-default-t8nc2" (UID: "b9492b29-8d64-43ca-9f70-250f32707cfd") : secret "dns-default-metrics-tls" not found Jun 08 15:34:07.496670 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:07.496638 2557 generic.go:356] "Generic (PLEG): container finished" podID="a7cad45c-9b89-43d6-bd57-7b3194b1d7c3" containerID="99057c5f1ecbb5238847b9f93527bac423c9e110fc72834db1f84cd7181fc387" exitCode=0 Jun 08 15:34:07.497138 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:07.496696 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tpwvw" event={"ID":"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3","Type":"ContainerDied","Data":"99057c5f1ecbb5238847b9f93527bac423c9e110fc72834db1f84cd7181fc387"} Jun 08 15:34:08.464191 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:08.464136 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:08.464374 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:08.464299 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 15:34:08.464374 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:08.464321 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f8f6f6b6-jx5jx: secret "image-registry-tls" not found Jun 08 15:34:08.464442 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:08.464377 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls podName:a0a2ada9-1340-40f3-b339-f2a682aecf8d nodeName:}" failed. No retries permitted until 2026-06-08 15:34:12.464361772 +0000 UTC m=+42.553492801 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls") pod "image-registry-84f8f6f6b6-jx5jx" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d") : secret "image-registry-tls" not found Jun 08 15:34:08.501663 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:08.501633 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tpwvw" event={"ID":"a7cad45c-9b89-43d6-bd57-7b3194b1d7c3","Type":"ContainerStarted","Data":"7fa94e9e0d03dcdef555d98739f29fa1de7c9e22c5cbcb519b6a1b5206808091"} Jun 08 15:34:08.506042 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:08.506008 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mvlgf" event={"ID":"ac6d6aa0-340a-4156-aa2d-60444b6f4fcb","Type":"ContainerStarted","Data":"3ec273c99c8f4b7ca14977e8b0f466e52d8134021d84b5d6edf297d561e98aea"} Jun 08 15:34:08.522133 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:08.522088 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tpwvw" podStartSLOduration=8.684326426 podStartE2EDuration="38.522073482s" totalImagesPullingTime="29.837747056s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 15:33:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:34:08.521769501 +0000 UTC m=+38.610900551" watchObservedRunningTime="2026-06-08 15:34:08.522073482 +0000 UTC m=+38.611204534" Jun 08 15:34:08.535191 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:08.535153 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mvlgf" podStartSLOduration=2.47633816 podStartE2EDuration="4.535140656s" totalImagesPullingTime="2.058802496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:34:08.534575745 +0000 UTC m=+38.623706796" watchObservedRunningTime="2026-06-08 15:34:08.535140656 +0000 UTC m=+38.624271706" Jun 08 15:34:08.564851 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:08.564823 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:08.564988 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:08.564968 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 15:34:08.565044 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:08.565027 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls podName:b9492b29-8d64-43ca-9f70-250f32707cfd nodeName:}" failed. No retries permitted until 2026-06-08 15:34:12.565013758 +0000 UTC m=+42.654144786 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls") pod "dns-default-t8nc2" (UID: "b9492b29-8d64-43ca-9f70-250f32707cfd") : secret "dns-default-metrics-tls" not found Jun 08 15:34:08.646115 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:08.646085 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:08.746135 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:08.746073 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:09.123385 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:09.123319 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:09.221096 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:09.221066 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:09.848044 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:09.848008 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:09.948678 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:09.948643 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:10.853226 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:10.853182 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:10.898961 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:10.898932 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:11.813243 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:11.813209 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:12.494394 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:12.494347 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:12.494811 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:12.494538 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 15:34:12.494811 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:12.494558 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f8f6f6b6-jx5jx: secret "image-registry-tls" not found Jun 08 15:34:12.494811 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:12.494612 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls podName:a0a2ada9-1340-40f3-b339-f2a682aecf8d nodeName:}" failed. No retries permitted until 2026-06-08 15:34:20.494596923 +0000 UTC m=+50.583727952 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls") pod "image-registry-84f8f6f6b6-jx5jx" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d") : secret "image-registry-tls" not found Jun 08 15:34:12.595402 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:12.595365 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:12.595568 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:12.595534 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 15:34:12.595608 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:12.595599 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls podName:b9492b29-8d64-43ca-9f70-250f32707cfd nodeName:}" failed. No retries permitted until 2026-06-08 15:34:20.595586006 +0000 UTC m=+50.684717038 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls") pod "dns-default-t8nc2" (UID: "b9492b29-8d64-43ca-9f70-250f32707cfd") : secret "dns-default-metrics-tls" not found Jun 08 15:34:16.809525 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:16.809492 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:16.911945 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:16.911912 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:16.962187 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:16.962146 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:17.290624 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:17.290567 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:17.336648 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:17.336619 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:17.871321 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:17.871289 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:17.970965 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:17.970936 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:19.811268 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:19.811230 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:20.551064 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:20.551035 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:20.551304 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:20.551136 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 15:34:20.551304 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:20.551148 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f8f6f6b6-jx5jx: secret "image-registry-tls" not found Jun 08 15:34:20.551304 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:20.551204 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls podName:a0a2ada9-1340-40f3-b339-f2a682aecf8d nodeName:}" failed. No retries permitted until 2026-06-08 15:34:36.55119078 +0000 UTC m=+66.640321807 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls") pod "image-registry-84f8f6f6b6-jx5jx" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d") : secret "image-registry-tls" not found Jun 08 15:34:20.652070 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:20.652029 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:20.652219 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:20.652200 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 15:34:20.652287 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:20.652276 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls podName:b9492b29-8d64-43ca-9f70-250f32707cfd nodeName:}" failed. No retries permitted until 2026-06-08 15:34:36.652255343 +0000 UTC m=+66.741386390 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls") pod "dns-default-t8nc2" (UID: "b9492b29-8d64-43ca-9f70-250f32707cfd") : secret "dns-default-metrics-tls" not found Jun 08 15:34:22.083155 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:22.083121 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:22.147320 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:22.147289 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:23.423860 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.423826 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4"] Jun 08 15:34:23.465883 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.465854 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4"] Jun 08 15:34:23.466024 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.465947 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:34:23.468317 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.468293 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 08 15:34:23.468963 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.468940 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 08 15:34:23.468963 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.468951 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 08 15:34:23.469129 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.468940 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 08 15:34:23.572909 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.572877 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hccd\" (UniqueName: \"kubernetes.io/projected/bb4ff32f-3b7d-4b34-9637-826937bd8b4a-kube-api-access-8hccd\") pod \"klusterlet-addon-workmgr-77776c944d-f6bd4\" (UID: \"bb4ff32f-3b7d-4b34-9637-826937bd8b4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:34:23.573066 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.572944 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bb4ff32f-3b7d-4b34-9637-826937bd8b4a-klusterlet-config\") pod \"klusterlet-addon-workmgr-77776c944d-f6bd4\" (UID: \"bb4ff32f-3b7d-4b34-9637-826937bd8b4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:34:23.573066 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.572995 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb4ff32f-3b7d-4b34-9637-826937bd8b4a-tmp\") pod \"klusterlet-addon-workmgr-77776c944d-f6bd4\" (UID: \"bb4ff32f-3b7d-4b34-9637-826937bd8b4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:34:23.674285 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.674223 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb4ff32f-3b7d-4b34-9637-826937bd8b4a-tmp\") pod \"klusterlet-addon-workmgr-77776c944d-f6bd4\" (UID: \"bb4ff32f-3b7d-4b34-9637-826937bd8b4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:34:23.674285 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.674254 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hccd\" (UniqueName: \"kubernetes.io/projected/bb4ff32f-3b7d-4b34-9637-826937bd8b4a-kube-api-access-8hccd\") pod \"klusterlet-addon-workmgr-77776c944d-f6bd4\" (UID: \"bb4ff32f-3b7d-4b34-9637-826937bd8b4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:34:23.674488 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.674297 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bb4ff32f-3b7d-4b34-9637-826937bd8b4a-klusterlet-config\") pod \"klusterlet-addon-workmgr-77776c944d-f6bd4\" (UID: \"bb4ff32f-3b7d-4b34-9637-826937bd8b4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:34:23.674617 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.674595 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb4ff32f-3b7d-4b34-9637-826937bd8b4a-tmp\") pod \"klusterlet-addon-workmgr-77776c944d-f6bd4\" (UID: \"bb4ff32f-3b7d-4b34-9637-826937bd8b4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:34:23.678103 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.678080 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bb4ff32f-3b7d-4b34-9637-826937bd8b4a-klusterlet-config\") pod \"klusterlet-addon-workmgr-77776c944d-f6bd4\" (UID: \"bb4ff32f-3b7d-4b34-9637-826937bd8b4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:34:23.682538 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.682514 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hccd\" (UniqueName: \"kubernetes.io/projected/bb4ff32f-3b7d-4b34-9637-826937bd8b4a-kube-api-access-8hccd\") pod \"klusterlet-addon-workmgr-77776c944d-f6bd4\" (UID: \"bb4ff32f-3b7d-4b34-9637-826937bd8b4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:34:23.774771 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.774749 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:34:23.912228 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:23.912196 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4"] Jun 08 15:34:23.916252 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:34:23.916225 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb4ff32f_3b7d_4b34_9637_826937bd8b4a.slice/crio-75687719246a8e3e72e1790fe7e739aa8a726e527da6ea4dd6d7805e29d49321 WatchSource:0}: Error finding container 75687719246a8e3e72e1790fe7e739aa8a726e527da6ea4dd6d7805e29d49321: Status 404 returned error can't find the container with id 75687719246a8e3e72e1790fe7e739aa8a726e527da6ea4dd6d7805e29d49321 Jun 08 15:34:24.541280 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:24.541247 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" event={"ID":"bb4ff32f-3b7d-4b34-9637-826937bd8b4a","Type":"ContainerStarted","Data":"75687719246a8e3e72e1790fe7e739aa8a726e527da6ea4dd6d7805e29d49321"} Jun 08 15:34:26.015513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:26.015475 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:26.066204 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:26.066014 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:26.936479 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:26.936433 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:26.995105 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:26.995067 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:28.077056 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:28.077026 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:28.177198 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:28.177163 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:28.554818 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:28.554786 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" event={"ID":"bb4ff32f-3b7d-4b34-9637-826937bd8b4a","Type":"ContainerStarted","Data":"6c21fdc5686867b5b510736bf8732043101d1cd540762d8968b53f1e7f6121f2"} Jun 08 15:34:28.555029 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:28.555004 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:34:28.556644 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:28.556627 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:34:28.582002 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:28.581955 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" podStartSLOduration=1.384120591 podStartE2EDuration="5.581937827s" totalImagesPullingTime="4.197817236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:34:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:34:28.568871344 +0000 UTC m=+58.658002394" watchObservedRunningTime="2026-06-08 15:34:28.581937827 +0000 UTC m=+58.671068879" Jun 08 15:34:30.481296 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:30.481261 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tsb88" Jun 08 15:34:30.630106 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:30.630071 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:30.730862 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:30.730835 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:32.944199 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:32.944162 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:32.996165 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:32.996139 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:33.047727 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:33.047693 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:36.070521 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.070451 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs\") pod \"network-metrics-daemon-hct6z\" (UID: \"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58\") " pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:34:36.070521 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.070524 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret\") pod \"global-pull-secret-syncer-r2mpb\" (UID: \"1e9d36fd-7367-43bf-8a04-3675efc9957d\") " pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:34:36.072668 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.072650 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 08 15:34:36.072787 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.072771 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 08 15:34:36.081256 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:36.081236 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 15:34:36.081339 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:36.081291 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs podName:75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58 nodeName:}" failed. No retries permitted until 2026-06-08 15:35:40.081276411 +0000 UTC m=+130.170407442 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs") pod "network-metrics-daemon-hct6z" (UID: "75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58") : secret "metrics-daemon-secret" not found Jun 08 15:34:36.083295 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.083279 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1e9d36fd-7367-43bf-8a04-3675efc9957d-original-pull-secret\") pod \"global-pull-secret-syncer-r2mpb\" (UID: \"1e9d36fd-7367-43bf-8a04-3675efc9957d\") " pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:34:36.171036 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.170999 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5krxh\" (UniqueName: \"kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh\") pod \"network-check-target-mvb2c\" (UID: \"7711bd0d-d822-4d5d-b62f-84f831446da0\") " pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:34:36.173548 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.173532 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 08 15:34:36.183684 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.183666 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 08 15:34:36.194272 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.194253 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5krxh\" (UniqueName: \"kubernetes.io/projected/7711bd0d-d822-4d5d-b62f-84f831446da0-kube-api-access-5krxh\") pod \"network-check-target-mvb2c\" (UID: \"7711bd0d-d822-4d5d-b62f-84f831446da0\") " pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:34:36.251637 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.251606 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r2mpb" Jun 08 15:34:36.258674 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.258657 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ld5bv\"" Jun 08 15:34:36.266762 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.266741 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:34:36.376249 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.376221 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-r2mpb"] Jun 08 15:34:36.380113 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:34:36.380090 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e9d36fd_7367_43bf_8a04_3675efc9957d.slice/crio-45ce8955ce53fb855c7d4253cc2644494738c5dfe429e2fd3840aa1259d2e7a5 WatchSource:0}: Error finding container 45ce8955ce53fb855c7d4253cc2644494738c5dfe429e2fd3840aa1259d2e7a5: Status 404 returned error can't find the container with id 45ce8955ce53fb855c7d4253cc2644494738c5dfe429e2fd3840aa1259d2e7a5 Jun 08 15:34:36.392561 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.392533 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mvb2c"] Jun 08 15:34:36.395747 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:34:36.395719 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7711bd0d_d822_4d5d_b62f_84f831446da0.slice/crio-9ffee79090aaa38512298cb4c4248ef20b2d2d13c226c73b2848bfc2eadf40ec WatchSource:0}: Error finding container 9ffee79090aaa38512298cb4c4248ef20b2d2d13c226c73b2848bfc2eadf40ec: Status 404 returned error can't find the container with id 9ffee79090aaa38512298cb4c4248ef20b2d2d13c226c73b2848bfc2eadf40ec Jun 08 15:34:36.572655 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.572620 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mvb2c" event={"ID":"7711bd0d-d822-4d5d-b62f-84f831446da0","Type":"ContainerStarted","Data":"9ffee79090aaa38512298cb4c4248ef20b2d2d13c226c73b2848bfc2eadf40ec"} Jun 08 15:34:36.573680 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.573654 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:34:36.573813 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.573771 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-r2mpb" event={"ID":"1e9d36fd-7367-43bf-8a04-3675efc9957d","Type":"ContainerStarted","Data":"45ce8955ce53fb855c7d4253cc2644494738c5dfe429e2fd3840aa1259d2e7a5"} Jun 08 15:34:36.573813 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:36.573800 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 15:34:36.573888 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:36.573814 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f8f6f6b6-jx5jx: secret "image-registry-tls" not found Jun 08 15:34:36.573888 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:36.573864 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls podName:a0a2ada9-1340-40f3-b339-f2a682aecf8d nodeName:}" failed. No retries permitted until 2026-06-08 15:35:08.573849671 +0000 UTC m=+98.662980702 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls") pod "image-registry-84f8f6f6b6-jx5jx" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d") : secret "image-registry-tls" not found Jun 08 15:34:36.674609 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:36.674584 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:34:36.674743 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:36.674699 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 15:34:36.674785 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:34:36.674748 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls podName:b9492b29-8d64-43ca-9f70-250f32707cfd nodeName:}" failed. No retries permitted until 2026-06-08 15:35:08.674734679 +0000 UTC m=+98.763865707 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls") pod "dns-default-t8nc2" (UID: "b9492b29-8d64-43ca-9f70-250f32707cfd") : secret "dns-default-metrics-tls" not found Jun 08 15:34:41.589686 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:41.589649 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mvb2c" event={"ID":"7711bd0d-d822-4d5d-b62f-84f831446da0","Type":"ContainerStarted","Data":"32ec6f819b382efd73423f8a717914ff8f875b94951c88e9d67f4089c0d0813b"} Jun 08 15:34:41.590131 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:41.589734 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:34:41.591030 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:41.591009 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-r2mpb" event={"ID":"1e9d36fd-7367-43bf-8a04-3675efc9957d","Type":"ContainerStarted","Data":"1ad720d8907ce67dd4c9785d6eee14a69bc08b3c77e070ce093f9be2899456b3"} Jun 08 15:34:41.604726 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:41.604688 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mvb2c" podStartSLOduration=67.217302535 podStartE2EDuration="1m11.604676806s" totalImagesPullingTime="4.387374271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:34:41.604016879 +0000 UTC m=+71.693147939" watchObservedRunningTime="2026-06-08 15:34:41.604676806 +0000 UTC m=+71.693807856" Jun 08 15:34:41.617016 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:41.616972 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-r2mpb" podStartSLOduration=67.178976688 podStartE2EDuration="1m11.616957492s" totalImagesPullingTime="4.437980804s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 15:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:34:41.616396779 +0000 UTC m=+71.705527820" watchObservedRunningTime="2026-06-08 15:34:41.616957492 +0000 UTC m=+71.706088545" Jun 08 15:34:46.916298 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:46.916264 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:46.968177 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:46.968143 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:54.887616 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:54.887571 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:54.943159 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:54.943133 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:34:54.994511 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:34:54.994479 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:35:08.604657 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:35:08.604623 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:35:08.605124 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:35:08.604740 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 15:35:08.605124 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:35:08.604752 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f8f6f6b6-jx5jx: secret "image-registry-tls" not found Jun 08 15:35:08.605124 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:35:08.604821 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls podName:a0a2ada9-1340-40f3-b339-f2a682aecf8d nodeName:}" failed. No retries permitted until 2026-06-08 15:36:12.604799577 +0000 UTC m=+162.693930604 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls") pod "image-registry-84f8f6f6b6-jx5jx" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d") : secret "image-registry-tls" not found Jun 08 15:35:08.705579 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:35:08.705555 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:35:08.705690 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:35:08.705678 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 15:35:08.705737 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:35:08.705729 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls podName:b9492b29-8d64-43ca-9f70-250f32707cfd nodeName:}" failed. No retries permitted until 2026-06-08 15:36:12.705716765 +0000 UTC m=+162.794847793 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls") pod "dns-default-t8nc2" (UID: "b9492b29-8d64-43ca-9f70-250f32707cfd") : secret "dns-default-metrics-tls" not found Jun 08 15:35:11.523041 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:35:11.522994 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:35:12.595843 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:35:12.595814 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mvb2c" Jun 08 15:35:35.653323 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:35:35.653259 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:35:40.119342 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:35:40.119316 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs\") pod \"network-metrics-daemon-hct6z\" (UID: \"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58\") " pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:35:40.119801 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:35:40.119451 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 15:35:40.119801 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:35:40.119535 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs podName:75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58 nodeName:}" failed. No retries permitted until 2026-06-08 15:37:42.119516963 +0000 UTC m=+252.208647993 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs") pod "network-metrics-daemon-hct6z" (UID: "75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58") : secret "metrics-daemon-secret" not found Jun 08 15:36:07.773142 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:07.773088 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" podUID="a0a2ada9-1340-40f3-b339-f2a682aecf8d" Jun 08 15:36:07.799777 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:07.799743 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-t8nc2" podUID="b9492b29-8d64-43ca-9f70-250f32707cfd" Jun 08 15:36:08.344524 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:08.344489 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-hct6z" podUID="75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58" Jun 08 15:36:12.650717 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:12.650676 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:36:12.651188 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:12.650826 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 15:36:12.651188 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:12.650842 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f8f6f6b6-jx5jx: secret "image-registry-tls" not found Jun 08 15:36:12.651188 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:12.650899 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls podName:a0a2ada9-1340-40f3-b339-f2a682aecf8d nodeName:}" failed. No retries permitted until 2026-06-08 15:38:14.650881689 +0000 UTC m=+284.740012736 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls") pod "image-registry-84f8f6f6b6-jx5jx" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d") : secret "image-registry-tls" not found Jun 08 15:36:12.751712 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:12.751678 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:36:12.751882 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:12.751804 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 15:36:12.751882 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:12.751877 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls podName:b9492b29-8d64-43ca-9f70-250f32707cfd nodeName:}" failed. No retries permitted until 2026-06-08 15:38:14.751858792 +0000 UTC m=+284.840989822 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls") pod "dns-default-t8nc2" (UID: "b9492b29-8d64-43ca-9f70-250f32707cfd") : secret "dns-default-metrics-tls" not found Jun 08 15:36:22.332633 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:22.332548 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:36:22.332989 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:22.332548 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t8nc2" Jun 08 15:36:22.332989 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:22.332548 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:36:24.911907 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:24.911873 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:28.556237 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:28.556148 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" podUID="bb4ff32f-3b7d-4b34-9637-826937bd8b4a" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.9:8000/readyz\": dial tcp 10.132.0.9:8000: connect: connection refused" Jun 08 15:36:28.846957 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:28.846877 2557 generic.go:356] "Generic (PLEG): container finished" podID="bb4ff32f-3b7d-4b34-9637-826937bd8b4a" containerID="6c21fdc5686867b5b510736bf8732043101d1cd540762d8968b53f1e7f6121f2" exitCode=1 Jun 08 15:36:28.846957 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:28.846915 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" event={"ID":"bb4ff32f-3b7d-4b34-9637-826937bd8b4a","Type":"ContainerDied","Data":"6c21fdc5686867b5b510736bf8732043101d1cd540762d8968b53f1e7f6121f2"} Jun 08 15:36:28.847267 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:28.847250 2557 scope.go:117] "RemoveContainer" containerID="6c21fdc5686867b5b510736bf8732043101d1cd540762d8968b53f1e7f6121f2" Jun 08 15:36:29.851893 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:29.851856 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" event={"ID":"bb4ff32f-3b7d-4b34-9637-826937bd8b4a","Type":"ContainerStarted","Data":"f196b5ba4e03a1c91107d7876ec099e69472823f79c702c90260f6523285fbaf"} Jun 08 15:36:29.852274 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:29.852205 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:36:29.852797 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:29.852777 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:36:48.283550 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.283515 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-hhh67"] Jun 08 15:36:48.287090 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.287047 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:36:48.287631 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.287602 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-664d9c4559-shg98"] Jun 08 15:36:48.290103 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.290084 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:48.290734 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.290711 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 08 15:36:48.290902 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.290887 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 08 15:36:48.291023 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.291001 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 08 15:36:48.291094 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.291053 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-cfth8\"" Jun 08 15:36:48.291157 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.291143 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 08 15:36:48.291932 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.291913 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 08 15:36:48.292403 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.292379 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 08 15:36:48.292716 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.292701 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 08 15:36:48.292813 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.292763 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 08 15:36:48.292916 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.292895 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 08 15:36:48.294247 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.294196 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 08 15:36:48.294338 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.294275 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vmkkq\"" Jun 08 15:36:48.297880 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.297861 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-hhh67"] Jun 08 15:36:48.299742 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.299722 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 08 15:36:48.305101 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.305080 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-664d9c4559-shg98"] Jun 08 15:36:48.379939 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.379914 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp"] Jun 08 15:36:48.381776 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.381746 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2q54\" (UniqueName: \"kubernetes.io/projected/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-kube-api-access-c2q54\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:48.381776 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.381758 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp" Jun 08 15:36:48.381938 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.381782 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfvvc\" (UniqueName: \"kubernetes.io/projected/2c03b1b7-10b4-43a0-a952-bd191d47aff5-kube-api-access-mfvvc\") pod \"console-operator-668fdbf85c-hhh67\" (UID: \"2c03b1b7-10b4-43a0-a952-bd191d47aff5\") " pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:36:48.381938 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.381815 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2c03b1b7-10b4-43a0-a952-bd191d47aff5-serving-cert\") pod \"console-operator-668fdbf85c-hhh67\" (UID: \"2c03b1b7-10b4-43a0-a952-bd191d47aff5\") " pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:36:48.381938 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.381835 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:48.381938 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.381856 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:48.381938 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.381891 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2c03b1b7-10b4-43a0-a952-bd191d47aff5-config\") pod \"console-operator-668fdbf85c-hhh67\" (UID: \"2c03b1b7-10b4-43a0-a952-bd191d47aff5\") " pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:36:48.381938 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.381938 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-default-certificate\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:48.382170 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.381961 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-stats-auth\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:48.382170 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.382022 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2c03b1b7-10b4-43a0-a952-bd191d47aff5-trusted-ca\") pod \"console-operator-668fdbf85c-hhh67\" (UID: \"2c03b1b7-10b4-43a0-a952-bd191d47aff5\") " pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:36:48.384674 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.384639 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-x9dmr\"" Jun 08 15:36:48.384852 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.384837 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jun 08 15:36:48.384963 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.384869 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jun 08 15:36:48.384963 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.384921 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jun 08 15:36:48.385179 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.385163 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jun 08 15:36:48.393261 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.393231 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp"] Jun 08 15:36:48.434925 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.434868 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:48.482648 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.482623 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:48.482751 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.482656 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2c03b1b7-10b4-43a0-a952-bd191d47aff5-config\") pod \"console-operator-668fdbf85c-hhh67\" (UID: \"2c03b1b7-10b4-43a0-a952-bd191d47aff5\") " pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:36:48.482751 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.482676 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-stats-auth\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:48.482751 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.482700 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/853db474-a505-43a0-8f8d-fb060c8bcc70-config\") pod \"service-ca-operator-6bc5ff68f5-9pkwp\" (UID: \"853db474-a505-43a0-8f8d-fb060c8bcc70\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp" Jun 08 15:36:48.482751 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.482736 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfvvc\" (UniqueName: \"kubernetes.io/projected/2c03b1b7-10b4-43a0-a952-bd191d47aff5-kube-api-access-mfvvc\") pod \"console-operator-668fdbf85c-hhh67\" (UID: \"2c03b1b7-10b4-43a0-a952-bd191d47aff5\") " pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:36:48.482974 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.482766 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv6r7\" (UniqueName: \"kubernetes.io/projected/853db474-a505-43a0-8f8d-fb060c8bcc70-kube-api-access-tv6r7\") pod \"service-ca-operator-6bc5ff68f5-9pkwp\" (UID: \"853db474-a505-43a0-8f8d-fb060c8bcc70\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp" Jun 08 15:36:48.482974 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:48.482779 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle podName:ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50 nodeName:}" failed. No retries permitted until 2026-06-08 15:36:48.982760798 +0000 UTC m=+199.071891826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle") pod "router-default-664d9c4559-shg98" (UID: "ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50") : configmap references non-existent config key: service-ca.crt Jun 08 15:36:48.482974 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.482846 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:48.482974 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.482875 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/853db474-a505-43a0-8f8d-fb060c8bcc70-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-9pkwp\" (UID: \"853db474-a505-43a0-8f8d-fb060c8bcc70\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp" Jun 08 15:36:48.482974 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.482915 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-default-certificate\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:48.482974 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.482951 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2c03b1b7-10b4-43a0-a952-bd191d47aff5-trusted-ca\") pod \"console-operator-668fdbf85c-hhh67\" (UID: \"2c03b1b7-10b4-43a0-a952-bd191d47aff5\") " pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:36:48.483333 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.483003 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2q54\" (UniqueName: \"kubernetes.io/projected/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-kube-api-access-c2q54\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:48.483333 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:48.482954 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 15:36:48.483333 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.483053 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2c03b1b7-10b4-43a0-a952-bd191d47aff5-serving-cert\") pod \"console-operator-668fdbf85c-hhh67\" (UID: \"2c03b1b7-10b4-43a0-a952-bd191d47aff5\") " pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:36:48.483333 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:48.483081 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs podName:ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50 nodeName:}" failed. No retries permitted until 2026-06-08 15:36:48.983061381 +0000 UTC m=+199.072192411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs") pod "router-default-664d9c4559-shg98" (UID: "ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50") : secret "router-metrics-certs-default" not found Jun 08 15:36:48.483333 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.483309 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2c03b1b7-10b4-43a0-a952-bd191d47aff5-config\") pod \"console-operator-668fdbf85c-hhh67\" (UID: \"2c03b1b7-10b4-43a0-a952-bd191d47aff5\") " pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:36:48.484035 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.484012 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2c03b1b7-10b4-43a0-a952-bd191d47aff5-trusted-ca\") pod \"console-operator-668fdbf85c-hhh67\" (UID: \"2c03b1b7-10b4-43a0-a952-bd191d47aff5\") " pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:36:48.485166 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.485145 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-stats-auth\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:48.485258 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.485169 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-default-certificate\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:48.485258 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.485249 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2c03b1b7-10b4-43a0-a952-bd191d47aff5-serving-cert\") pod \"console-operator-668fdbf85c-hhh67\" (UID: \"2c03b1b7-10b4-43a0-a952-bd191d47aff5\") " pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:36:48.490560 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.490453 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2q54\" (UniqueName: \"kubernetes.io/projected/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-kube-api-access-c2q54\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:48.491025 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.491002 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfvvc\" (UniqueName: \"kubernetes.io/projected/2c03b1b7-10b4-43a0-a952-bd191d47aff5-kube-api-access-mfvvc\") pod \"console-operator-668fdbf85c-hhh67\" (UID: \"2c03b1b7-10b4-43a0-a952-bd191d47aff5\") " pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:36:48.536491 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.536406 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:48.583632 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.583602 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv6r7\" (UniqueName: \"kubernetes.io/projected/853db474-a505-43a0-8f8d-fb060c8bcc70-kube-api-access-tv6r7\") pod \"service-ca-operator-6bc5ff68f5-9pkwp\" (UID: \"853db474-a505-43a0-8f8d-fb060c8bcc70\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp" Jun 08 15:36:48.583757 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.583651 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/853db474-a505-43a0-8f8d-fb060c8bcc70-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-9pkwp\" (UID: \"853db474-a505-43a0-8f8d-fb060c8bcc70\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp" Jun 08 15:36:48.583846 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.583827 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/853db474-a505-43a0-8f8d-fb060c8bcc70-config\") pod \"service-ca-operator-6bc5ff68f5-9pkwp\" (UID: \"853db474-a505-43a0-8f8d-fb060c8bcc70\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp" Jun 08 15:36:48.584257 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.584238 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/853db474-a505-43a0-8f8d-fb060c8bcc70-config\") pod \"service-ca-operator-6bc5ff68f5-9pkwp\" (UID: \"853db474-a505-43a0-8f8d-fb060c8bcc70\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp" Jun 08 15:36:48.585742 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.585723 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/853db474-a505-43a0-8f8d-fb060c8bcc70-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-9pkwp\" (UID: \"853db474-a505-43a0-8f8d-fb060c8bcc70\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp" Jun 08 15:36:48.591389 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.591369 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv6r7\" (UniqueName: \"kubernetes.io/projected/853db474-a505-43a0-8f8d-fb060c8bcc70-kube-api-access-tv6r7\") pod \"service-ca-operator-6bc5ff68f5-9pkwp\" (UID: \"853db474-a505-43a0-8f8d-fb060c8bcc70\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp" Jun 08 15:36:48.599310 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.599291 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:36:48.689907 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.689879 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp" Jun 08 15:36:48.719557 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.719517 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-hhh67"] Jun 08 15:36:48.723595 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:36:48.723557 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c03b1b7_10b4_43a0_a952_bd191d47aff5.slice/crio-1aecdd169aa9bf028dd21dfb86a0f2eb81be8a26c4d45ef93feb4e4db0971a64 WatchSource:0}: Error finding container 1aecdd169aa9bf028dd21dfb86a0f2eb81be8a26c4d45ef93feb4e4db0971a64: Status 404 returned error can't find the container with id 1aecdd169aa9bf028dd21dfb86a0f2eb81be8a26c4d45ef93feb4e4db0971a64 Jun 08 15:36:48.807635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.807542 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp"] Jun 08 15:36:48.810380 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:36:48.810350 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853db474_a505_43a0_8f8d_fb060c8bcc70.slice/crio-7a4f17d9461ed849f8d49b19670cfccfb03dd41ded0b274714732d0f9f78257b WatchSource:0}: Error finding container 7a4f17d9461ed849f8d49b19670cfccfb03dd41ded0b274714732d0f9f78257b: Status 404 returned error can't find the container with id 7a4f17d9461ed849f8d49b19670cfccfb03dd41ded0b274714732d0f9f78257b Jun 08 15:36:48.905890 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.905862 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" event={"ID":"2c03b1b7-10b4-43a0-a952-bd191d47aff5","Type":"ContainerStarted","Data":"1aecdd169aa9bf028dd21dfb86a0f2eb81be8a26c4d45ef93feb4e4db0971a64"} Jun 08 15:36:48.906837 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.906807 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp" event={"ID":"853db474-a505-43a0-8f8d-fb060c8bcc70","Type":"ContainerStarted","Data":"7a4f17d9461ed849f8d49b19670cfccfb03dd41ded0b274714732d0f9f78257b"} Jun 08 15:36:48.988502 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.988458 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:48.988629 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:48.988566 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:48.988629 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:48.988601 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 15:36:48.988692 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:48.988658 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs podName:ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50 nodeName:}" failed. No retries permitted until 2026-06-08 15:36:49.988642491 +0000 UTC m=+200.077773518 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs") pod "router-default-664d9c4559-shg98" (UID: "ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50") : secret "router-metrics-certs-default" not found Jun 08 15:36:48.988692 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:48.988684 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle podName:ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50 nodeName:}" failed. No retries permitted until 2026-06-08 15:36:49.988672183 +0000 UTC m=+200.077803211 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle") pod "router-default-664d9c4559-shg98" (UID: "ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50") : configmap references non-existent config key: service-ca.crt Jun 08 15:36:49.998224 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:49.998179 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:49.998652 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:49.998295 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:49.998652 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:49.998326 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 15:36:49.998652 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:49.998388 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs podName:ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50 nodeName:}" failed. No retries permitted until 2026-06-08 15:36:51.998373285 +0000 UTC m=+202.087504313 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs") pod "router-default-664d9c4559-shg98" (UID: "ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50") : secret "router-metrics-certs-default" not found Jun 08 15:36:49.998652 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:49.998406 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle podName:ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50 nodeName:}" failed. No retries permitted until 2026-06-08 15:36:51.998399254 +0000 UTC m=+202.087530282 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle") pod "router-default-664d9c4559-shg98" (UID: "ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50") : configmap references non-existent config key: service-ca.crt Jun 08 15:36:50.213207 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:50.213163 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:50.266766 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:50.266675 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:51.918094 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:51.918066 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/0.log" Jun 08 15:36:51.918486 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:51.918106 2557 generic.go:356] "Generic (PLEG): container finished" podID="2c03b1b7-10b4-43a0-a952-bd191d47aff5" containerID="ab57febbe88ccf84ba0e43ec12c5b17dd8541c23c6b08dd781a315cf92e01969" exitCode=255 Jun 08 15:36:51.918486 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:51.918199 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" event={"ID":"2c03b1b7-10b4-43a0-a952-bd191d47aff5","Type":"ContainerDied","Data":"ab57febbe88ccf84ba0e43ec12c5b17dd8541c23c6b08dd781a315cf92e01969"} Jun 08 15:36:51.918486 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:51.918391 2557 scope.go:117] "RemoveContainer" containerID="ab57febbe88ccf84ba0e43ec12c5b17dd8541c23c6b08dd781a315cf92e01969" Jun 08 15:36:51.919812 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:51.919788 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp" event={"ID":"853db474-a505-43a0-8f8d-fb060c8bcc70","Type":"ContainerStarted","Data":"096bff9a3ecec7d1d71de5bfec4fdec6ec2f966e0b87f336823df47a6a35f5d2"} Jun 08 15:36:51.946080 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:51.946058 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:51.950500 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:51.950435 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp" podStartSLOduration=1.461531516 podStartE2EDuration="3.950419557s" totalImagesPullingTime="2.488888041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:36:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:36:51.949795901 +0000 UTC m=+202.038926974" watchObservedRunningTime="2026-06-08 15:36:51.950419557 +0000 UTC m=+202.039550609" Jun 08 15:36:52.015213 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:52.015174 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:52.015379 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:52.015320 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:52.015379 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:52.015350 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle podName:ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50 nodeName:}" failed. No retries permitted until 2026-06-08 15:36:56.015326498 +0000 UTC m=+206.104457546 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle") pod "router-default-664d9c4559-shg98" (UID: "ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50") : configmap references non-existent config key: service-ca.crt Jun 08 15:36:52.015523 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:52.015421 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 15:36:52.015523 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:52.015496 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs podName:ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50 nodeName:}" failed. No retries permitted until 2026-06-08 15:36:56.01545401 +0000 UTC m=+206.104585055 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs") pod "router-default-664d9c4559-shg98" (UID: "ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50") : secret "router-metrics-certs-default" not found Jun 08 15:36:52.090541 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:52.090509 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:52.380357 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:52.380326 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:52.473510 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:52.473482 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:52.574521 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:52.574484 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:52.924603 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:52.924576 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/1.log" Jun 08 15:36:52.925035 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:52.925019 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/0.log" Jun 08 15:36:52.925102 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:52.925064 2557 generic.go:356] "Generic (PLEG): container finished" podID="2c03b1b7-10b4-43a0-a952-bd191d47aff5" containerID="09532e3400b0330c87c29f3e33a321a6545267c0ff56c110e78dd89bef496e3e" exitCode=255 Jun 08 15:36:52.925164 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:52.925140 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" event={"ID":"2c03b1b7-10b4-43a0-a952-bd191d47aff5","Type":"ContainerDied","Data":"09532e3400b0330c87c29f3e33a321a6545267c0ff56c110e78dd89bef496e3e"} Jun 08 15:36:52.925209 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:52.925200 2557 scope.go:117] "RemoveContainer" containerID="ab57febbe88ccf84ba0e43ec12c5b17dd8541c23c6b08dd781a315cf92e01969" Jun 08 15:36:52.925436 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:52.925412 2557 scope.go:117] "RemoveContainer" containerID="09532e3400b0330c87c29f3e33a321a6545267c0ff56c110e78dd89bef496e3e" Jun 08 15:36:52.925664 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:52.925628 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-hhh67_openshift-console-operator(2c03b1b7-10b4-43a0-a952-bd191d47aff5)\"" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" podUID="2c03b1b7-10b4-43a0-a952-bd191d47aff5" Jun 08 15:36:53.021654 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:53.021622 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:53.121641 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:53.121603 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:53.723527 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:53.723494 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:53.825588 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:53.825552 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:53.929532 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:53.929508 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/1.log" Jun 08 15:36:53.929894 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:53.929821 2557 scope.go:117] "RemoveContainer" containerID="09532e3400b0330c87c29f3e33a321a6545267c0ff56c110e78dd89bef496e3e" Jun 08 15:36:53.929979 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:53.929958 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-hhh67_openshift-console-operator(2c03b1b7-10b4-43a0-a952-bd191d47aff5)\"" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" podUID="2c03b1b7-10b4-43a0-a952-bd191d47aff5" Jun 08 15:36:54.106360 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:54.106286 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l8rfj_9865e636-5527-48fe-b4a8-de43e6b507ec/dns-node-resolver/0.log" Jun 08 15:36:54.239356 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:54.239319 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:54.906663 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:54.906636 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5t7fb_d777d945-3cc7-4e04-b64b-f021f129aea7/node-ca/0.log" Jun 08 15:36:55.361309 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:55.361196 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:55.408791 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:55.408747 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:55.706829 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:55.706804 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mvlgf_ac6d6aa0-340a-4156-aa2d-60444b6f4fcb/serve-healthcheck-canary/0.log" Jun 08 15:36:56.043806 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:56.043727 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:56.043806 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:56.043780 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:36:56.043973 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:56.043884 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle podName:ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50 nodeName:}" failed. No retries permitted until 2026-06-08 15:37:04.043865485 +0000 UTC m=+214.132996515 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle") pod "router-default-664d9c4559-shg98" (UID: "ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50") : configmap references non-existent config key: service-ca.crt Jun 08 15:36:56.043973 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:56.043923 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 15:36:56.044045 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:56.043974 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs podName:ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50 nodeName:}" failed. No retries permitted until 2026-06-08 15:37:04.043960222 +0000 UTC m=+214.133091251 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs") pod "router-default-664d9c4559-shg98" (UID: "ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50") : secret "router-metrics-certs-default" not found Jun 08 15:36:56.243227 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:56.243196 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:58.600193 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:58.600162 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:36:58.600568 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:58.600221 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:36:58.600633 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:58.600572 2557 scope.go:117] "RemoveContainer" containerID="09532e3400b0330c87c29f3e33a321a6545267c0ff56c110e78dd89bef496e3e" Jun 08 15:36:58.600735 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:36:58.600717 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-hhh67_openshift-console-operator(2c03b1b7-10b4-43a0-a952-bd191d47aff5)\"" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" podUID="2c03b1b7-10b4-43a0-a952-bd191d47aff5" Jun 08 15:36:58.974805 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:58.974747 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:59.094208 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:59.094161 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:59.209372 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:59.209340 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:59.298348 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:59.298280 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:59.419872 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:59.419840 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:59.520512 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:59.520457 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:36:59.671168 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:36:59.671127 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:00.870186 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:00.870153 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:01.071657 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:01.071626 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:01.272152 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:01.272123 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:01.479193 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:01.479156 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:01.670947 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:01.670909 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:02.573162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:02.573119 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:02.673763 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:02.673728 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:02.723539 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:02.723509 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:02.864601 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:02.864515 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:03.069835 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:03.069800 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:03.268988 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:03.268952 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:03.469578 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:03.469538 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:03.870080 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:03.870025 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:04.108859 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:04.108816 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:37:04.109044 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:04.108886 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:37:04.109044 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:37:04.109016 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle podName:ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50 nodeName:}" failed. No retries permitted until 2026-06-08 15:37:20.108996177 +0000 UTC m=+230.198127222 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle") pod "router-default-664d9c4559-shg98" (UID: "ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50") : configmap references non-existent config key: service-ca.crt Jun 08 15:37:04.111299 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:04.111280 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-metrics-certs\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:37:04.469759 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:04.469715 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:04.868918 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:04.868840 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:05.044892 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:05.044856 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:05.269283 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:05.269247 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:05.470357 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:05.470324 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:05.524284 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:05.524210 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:06.065993 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:06.065937 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:06.273230 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:06.273192 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:06.668197 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:06.668163 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:07.470274 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:07.470224 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:07.877184 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:07.877155 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:08.068207 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:08.068165 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:08.271841 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:08.271762 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:08.670690 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:08.670662 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:09.276996 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:09.276971 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:09.470021 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:09.469988 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:09.667910 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:09.667884 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:14.332952 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:14.332922 2557 scope.go:117] "RemoveContainer" containerID="09532e3400b0330c87c29f3e33a321a6545267c0ff56c110e78dd89bef496e3e" Jun 08 15:37:14.991836 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:14.991805 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 15:37:14.992196 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:14.992180 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/1.log" Jun 08 15:37:14.992246 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:14.992212 2557 generic.go:356] "Generic (PLEG): container finished" podID="2c03b1b7-10b4-43a0-a952-bd191d47aff5" containerID="21513c1ab3e8546ea50f4d4eb0243e7007a460a717076e125a0010d540855ba4" exitCode=255 Jun 08 15:37:14.992285 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:14.992261 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" event={"ID":"2c03b1b7-10b4-43a0-a952-bd191d47aff5","Type":"ContainerDied","Data":"21513c1ab3e8546ea50f4d4eb0243e7007a460a717076e125a0010d540855ba4"} Jun 08 15:37:14.992329 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:14.992301 2557 scope.go:117] "RemoveContainer" containerID="09532e3400b0330c87c29f3e33a321a6545267c0ff56c110e78dd89bef496e3e" Jun 08 15:37:14.992629 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:14.992611 2557 scope.go:117] "RemoveContainer" containerID="21513c1ab3e8546ea50f4d4eb0243e7007a460a717076e125a0010d540855ba4" Jun 08 15:37:14.992816 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:37:14.992793 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-hhh67_openshift-console-operator(2c03b1b7-10b4-43a0-a952-bd191d47aff5)\"" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" podUID="2c03b1b7-10b4-43a0-a952-bd191d47aff5" Jun 08 15:37:15.638704 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.638673 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wgrqt"] Jun 08 15:37:15.641588 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.641564 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.643871 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.643851 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 08 15:37:15.644732 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.644713 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 08 15:37:15.644850 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.644739 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 08 15:37:15.644850 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.644784 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 08 15:37:15.644850 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.644827 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pz2pf\"" Jun 08 15:37:15.652830 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.652811 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wgrqt"] Jun 08 15:37:15.743752 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.743720 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:15.795885 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.795863 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/09ff7cba-a236-4931-97aa-0c52dc0e2ec3-data-volume\") pod \"insights-runtime-extractor-wgrqt\" (UID: \"09ff7cba-a236-4931-97aa-0c52dc0e2ec3\") " pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.795969 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.795932 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09ff7cba-a236-4931-97aa-0c52dc0e2ec3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wgrqt\" (UID: \"09ff7cba-a236-4931-97aa-0c52dc0e2ec3\") " pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.796015 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.795971 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/09ff7cba-a236-4931-97aa-0c52dc0e2ec3-crio-socket\") pod \"insights-runtime-extractor-wgrqt\" (UID: \"09ff7cba-a236-4931-97aa-0c52dc0e2ec3\") " pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.796015 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.795997 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09ff7cba-a236-4931-97aa-0c52dc0e2ec3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wgrqt\" (UID: \"09ff7cba-a236-4931-97aa-0c52dc0e2ec3\") " pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.796079 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.796031 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxjs5\" (UniqueName: \"kubernetes.io/projected/09ff7cba-a236-4931-97aa-0c52dc0e2ec3-kube-api-access-vxjs5\") pod \"insights-runtime-extractor-wgrqt\" (UID: \"09ff7cba-a236-4931-97aa-0c52dc0e2ec3\") " pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.844363 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.844331 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:15.896513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.896440 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/09ff7cba-a236-4931-97aa-0c52dc0e2ec3-crio-socket\") pod \"insights-runtime-extractor-wgrqt\" (UID: \"09ff7cba-a236-4931-97aa-0c52dc0e2ec3\") " pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.896513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.896499 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09ff7cba-a236-4931-97aa-0c52dc0e2ec3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wgrqt\" (UID: \"09ff7cba-a236-4931-97aa-0c52dc0e2ec3\") " pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.896686 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.896532 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxjs5\" (UniqueName: \"kubernetes.io/projected/09ff7cba-a236-4931-97aa-0c52dc0e2ec3-kube-api-access-vxjs5\") pod \"insights-runtime-extractor-wgrqt\" (UID: \"09ff7cba-a236-4931-97aa-0c52dc0e2ec3\") " pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.896686 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.896553 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/09ff7cba-a236-4931-97aa-0c52dc0e2ec3-data-volume\") pod \"insights-runtime-extractor-wgrqt\" (UID: \"09ff7cba-a236-4931-97aa-0c52dc0e2ec3\") " pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.896686 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.896574 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/09ff7cba-a236-4931-97aa-0c52dc0e2ec3-crio-socket\") pod \"insights-runtime-extractor-wgrqt\" (UID: \"09ff7cba-a236-4931-97aa-0c52dc0e2ec3\") " pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.896686 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.896600 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09ff7cba-a236-4931-97aa-0c52dc0e2ec3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wgrqt\" (UID: \"09ff7cba-a236-4931-97aa-0c52dc0e2ec3\") " pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.896938 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.896918 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/09ff7cba-a236-4931-97aa-0c52dc0e2ec3-data-volume\") pod \"insights-runtime-extractor-wgrqt\" (UID: \"09ff7cba-a236-4931-97aa-0c52dc0e2ec3\") " pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.897134 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.897112 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09ff7cba-a236-4931-97aa-0c52dc0e2ec3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wgrqt\" (UID: \"09ff7cba-a236-4931-97aa-0c52dc0e2ec3\") " pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.898940 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.898923 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09ff7cba-a236-4931-97aa-0c52dc0e2ec3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wgrqt\" (UID: \"09ff7cba-a236-4931-97aa-0c52dc0e2ec3\") " pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.903376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.903356 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxjs5\" (UniqueName: \"kubernetes.io/projected/09ff7cba-a236-4931-97aa-0c52dc0e2ec3-kube-api-access-vxjs5\") pod \"insights-runtime-extractor-wgrqt\" (UID: \"09ff7cba-a236-4931-97aa-0c52dc0e2ec3\") " pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.950684 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.950665 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wgrqt" Jun 08 15:37:15.998134 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:15.998109 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 15:37:16.068721 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:16.068692 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wgrqt"] Jun 08 15:37:16.071705 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:37:16.071672 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09ff7cba_a236_4931_97aa_0c52dc0e2ec3.slice/crio-1ee7175cda47aa05fd38839e9acf275933662525154b743bd2edca8bcd258731 WatchSource:0}: Error finding container 1ee7175cda47aa05fd38839e9acf275933662525154b743bd2edca8bcd258731: Status 404 returned error can't find the container with id 1ee7175cda47aa05fd38839e9acf275933662525154b743bd2edca8bcd258731 Jun 08 15:37:17.002367 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:17.002288 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wgrqt" event={"ID":"09ff7cba-a236-4931-97aa-0c52dc0e2ec3","Type":"ContainerStarted","Data":"d47abc1d320086a566e02566ebeb502d03c52a8f992a1cffb26f8fd2b3fd1eac"} Jun 08 15:37:17.002367 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:17.002340 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wgrqt" event={"ID":"09ff7cba-a236-4931-97aa-0c52dc0e2ec3","Type":"ContainerStarted","Data":"7d3c0606f818a45b3aca2f6ce0b221530e074f9b72e6996652a441b718d865f3"} Jun 08 15:37:17.002367 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:17.002360 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wgrqt" event={"ID":"09ff7cba-a236-4931-97aa-0c52dc0e2ec3","Type":"ContainerStarted","Data":"1ee7175cda47aa05fd38839e9acf275933662525154b743bd2edca8bcd258731"} Jun 08 15:37:17.289790 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:17.289704 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:18.520024 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:18.519924 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:18.599796 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:18.599745 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:37:18.599976 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:18.599819 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:37:18.600165 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:18.600150 2557 scope.go:117] "RemoveContainer" containerID="21513c1ab3e8546ea50f4d4eb0243e7007a460a717076e125a0010d540855ba4" Jun 08 15:37:18.600346 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:37:18.600327 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-hhh67_openshift-console-operator(2c03b1b7-10b4-43a0-a952-bd191d47aff5)\"" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" podUID="2c03b1b7-10b4-43a0-a952-bd191d47aff5" Jun 08 15:37:18.816005 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:18.815912 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:19.010890 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:19.010859 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wgrqt" event={"ID":"09ff7cba-a236-4931-97aa-0c52dc0e2ec3","Type":"ContainerStarted","Data":"b262e3a13093872d0ae0b3b24d96f1010e7c1adae97204bdbf4c893114564e45"} Jun 08 15:37:19.028573 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:19.028527 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wgrqt" podStartSLOduration=1.963773617 podStartE2EDuration="4.028511144s" totalImagesPullingTime="2.064737527s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 15:37:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:37:19.027525648 +0000 UTC m=+229.116656698" watchObservedRunningTime="2026-06-08 15:37:19.028511144 +0000 UTC m=+229.117642195" Jun 08 15:37:20.127534 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:20.127503 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:37:20.128070 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:20.128052 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50-service-ca-bundle\") pod \"router-default-664d9c4559-shg98\" (UID: \"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50\") " pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:37:20.404135 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:20.404044 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:37:20.524942 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:20.524918 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-664d9c4559-shg98"] Jun 08 15:37:20.527172 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:37:20.527138 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad9b4f9b_fe56_4de3_8c0c_0dd5ce50ce50.slice/crio-e10e1891c937db381f0296fad3daaafffd64299605efddc0ec956cf66610f587 WatchSource:0}: Error finding container e10e1891c937db381f0296fad3daaafffd64299605efddc0ec956cf66610f587: Status 404 returned error can't find the container with id e10e1891c937db381f0296fad3daaafffd64299605efddc0ec956cf66610f587 Jun 08 15:37:21.021338 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:21.021302 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-664d9c4559-shg98" event={"ID":"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50","Type":"ContainerStarted","Data":"18ca66b6b671c0304fee0f51ebf90a6b4c43b58d73d47621bf0152f1deab470c"} Jun 08 15:37:21.021572 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:21.021387 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-664d9c4559-shg98" event={"ID":"ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50","Type":"ContainerStarted","Data":"e10e1891c937db381f0296fad3daaafffd64299605efddc0ec956cf66610f587"} Jun 08 15:37:21.039082 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:21.039028 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-664d9c4559-shg98" podStartSLOduration=33.039011298 podStartE2EDuration="33.039011298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:36:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:37:21.038179802 +0000 UTC m=+231.127310852" watchObservedRunningTime="2026-06-08 15:37:21.039011298 +0000 UTC m=+231.128142350" Jun 08 15:37:21.405239 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:21.405208 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:37:21.407822 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:21.407800 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:37:22.024815 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:22.024781 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:37:22.025921 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:22.025896 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-664d9c4559-shg98" Jun 08 15:37:23.155592 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:23.155563 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:23.256548 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:23.256510 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:23.316186 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:23.316157 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-hqtpt"] Jun 08 15:37:23.318501 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:23.318455 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-hqtpt" Jun 08 15:37:23.319458 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:23.319432 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:23.320690 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:23.320671 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-84z2g\"" Jun 08 15:37:23.320796 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:23.320699 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 08 15:37:23.329209 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:23.329191 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-hqtpt"] Jun 08 15:37:23.345138 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:23.345111 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9d9afdc4-b5f0-4954-b58e-78441058f608-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-hqtpt\" (UID: \"9d9afdc4-b5f0-4954-b58e-78441058f608\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-hqtpt" Jun 08 15:37:23.409622 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:23.409550 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:23.445719 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:23.445698 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9d9afdc4-b5f0-4954-b58e-78441058f608-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-hqtpt\" (UID: \"9d9afdc4-b5f0-4954-b58e-78441058f608\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-hqtpt" Jun 08 15:37:23.445798 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:37:23.445785 2557 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jun 08 15:37:23.445844 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:37:23.445834 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d9afdc4-b5f0-4954-b58e-78441058f608-tls-certificates podName:9d9afdc4-b5f0-4954-b58e-78441058f608 nodeName:}" failed. No retries permitted until 2026-06-08 15:37:23.945821485 +0000 UTC m=+234.034952513 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/9d9afdc4-b5f0-4954-b58e-78441058f608-tls-certificates") pod "prometheus-operator-admission-webhook-5b6b8f594-hqtpt" (UID: "9d9afdc4-b5f0-4954-b58e-78441058f608") : secret "prometheus-operator-admission-webhook-tls" not found Jun 08 15:37:23.511182 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:23.511157 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:23.612437 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:23.612408 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:23.949245 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:23.949214 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9d9afdc4-b5f0-4954-b58e-78441058f608-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-hqtpt\" (UID: \"9d9afdc4-b5f0-4954-b58e-78441058f608\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-hqtpt" Jun 08 15:37:23.951423 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:23.951395 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9d9afdc4-b5f0-4954-b58e-78441058f608-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-hqtpt\" (UID: \"9d9afdc4-b5f0-4954-b58e-78441058f608\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-hqtpt" Jun 08 15:37:24.228138 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:24.228038 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-hqtpt" Jun 08 15:37:24.345636 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:24.345607 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-hqtpt"] Jun 08 15:37:24.349287 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:37:24.349252 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d9afdc4_b5f0_4954_b58e_78441058f608.slice/crio-bdd1e088005339585b9255674575d6929f6cb624dd2046e06d5be3cc9e42872f WatchSource:0}: Error finding container bdd1e088005339585b9255674575d6929f6cb624dd2046e06d5be3cc9e42872f: Status 404 returned error can't find the container with id bdd1e088005339585b9255674575d6929f6cb624dd2046e06d5be3cc9e42872f Jun 08 15:37:25.034981 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:25.034928 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-hqtpt" event={"ID":"9d9afdc4-b5f0-4954-b58e-78441058f608","Type":"ContainerStarted","Data":"bdd1e088005339585b9255674575d6929f6cb624dd2046e06d5be3cc9e42872f"} Jun 08 15:37:26.039056 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.039023 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-hqtpt" event={"ID":"9d9afdc4-b5f0-4954-b58e-78441058f608","Type":"ContainerStarted","Data":"4055cbd3f7a504489cbbe8c777aeb53c46cac03dee9389f9177163e71fd3876f"} Jun 08 15:37:26.039495 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.039312 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-hqtpt" Jun 08 15:37:26.043847 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.043825 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-hqtpt" Jun 08 15:37:26.053819 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.053781 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-hqtpt" podStartSLOduration=2.034239811 podStartE2EDuration="3.053768167s" totalImagesPullingTime="1.019528356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:37:26.05317601 +0000 UTC m=+236.142307070" watchObservedRunningTime="2026-06-08 15:37:26.053768167 +0000 UTC m=+236.142899216" Jun 08 15:37:26.342308 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.342220 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:26.366992 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.366962 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n"] Jun 08 15:37:26.372121 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.372095 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" Jun 08 15:37:26.374430 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.374408 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 08 15:37:26.374978 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.374960 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 08 15:37:26.374978 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.374980 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 08 15:37:26.375120 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.374969 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-5vvgg\"" Jun 08 15:37:26.375120 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.374982 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 08 15:37:26.375120 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.374965 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 08 15:37:26.379012 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.378971 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n"] Jun 08 15:37:26.397206 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.397182 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:26.448842 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.448816 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:26.470754 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.470729 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5zvv\" (UniqueName: \"kubernetes.io/projected/de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736-kube-api-access-z5zvv\") pod \"prometheus-operator-7f7d445d84-mqq5n\" (UID: \"de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" Jun 08 15:37:26.470836 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.470811 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-mqq5n\" (UID: \"de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" Jun 08 15:37:26.470836 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.470831 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-mqq5n\" (UID: \"de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" Jun 08 15:37:26.470924 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.470853 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-mqq5n\" (UID: \"de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" Jun 08 15:37:26.571760 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.571727 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5zvv\" (UniqueName: \"kubernetes.io/projected/de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736-kube-api-access-z5zvv\") pod \"prometheus-operator-7f7d445d84-mqq5n\" (UID: \"de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" Jun 08 15:37:26.571892 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.571814 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-mqq5n\" (UID: \"de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" Jun 08 15:37:26.571892 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.571836 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-mqq5n\" (UID: \"de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" Jun 08 15:37:26.571892 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.571858 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-mqq5n\" (UID: \"de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" Jun 08 15:37:26.572618 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.572596 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-mqq5n\" (UID: \"de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" Jun 08 15:37:26.574202 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.574182 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-mqq5n\" (UID: \"de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" Jun 08 15:37:26.574279 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.574236 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-mqq5n\" (UID: \"de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" Jun 08 15:37:26.578628 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.578602 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5zvv\" (UniqueName: \"kubernetes.io/projected/de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736-kube-api-access-z5zvv\") pod \"prometheus-operator-7f7d445d84-mqq5n\" (UID: \"de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" Jun 08 15:37:26.682761 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.682726 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" Jun 08 15:37:26.803023 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:26.802994 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n"] Jun 08 15:37:26.806743 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:37:26.806713 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde4cb47a_7cb5_4c7b_bc29_f9ab69fcc736.slice/crio-8505e60dc548ab9de3564689dfd348e4e1a5feb0f3bf785e1853c1d8621d95a8 WatchSource:0}: Error finding container 8505e60dc548ab9de3564689dfd348e4e1a5feb0f3bf785e1853c1d8621d95a8: Status 404 returned error can't find the container with id 8505e60dc548ab9de3564689dfd348e4e1a5feb0f3bf785e1853c1d8621d95a8 Jun 08 15:37:27.043284 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:27.043202 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" event={"ID":"de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736","Type":"ContainerStarted","Data":"8505e60dc548ab9de3564689dfd348e4e1a5feb0f3bf785e1853c1d8621d95a8"} Jun 08 15:37:28.323175 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:28.323092 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:29.052637 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:29.052595 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" event={"ID":"de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736","Type":"ContainerStarted","Data":"533d68185d00dd4160b7a1e487dbeb95f636f0272e350aa6295bf80bb17aa48c"} Jun 08 15:37:29.052832 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:29.052655 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" event={"ID":"de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736","Type":"ContainerStarted","Data":"64d7121872b4e3c055246540dcde8d4a22fb4be88209c914f29745b371db04f8"} Jun 08 15:37:29.069224 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:29.069177 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7f7d445d84-mqq5n" podStartSLOduration=1.814128822 podStartE2EDuration="3.069157212s" totalImagesPullingTime="1.25502839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:37:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:37:29.068549808 +0000 UTC m=+239.157680859" watchObservedRunningTime="2026-06-08 15:37:29.069157212 +0000 UTC m=+239.158288263" Jun 08 15:37:29.332757 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:29.332682 2557 scope.go:117] "RemoveContainer" containerID="21513c1ab3e8546ea50f4d4eb0243e7007a460a717076e125a0010d540855ba4" Jun 08 15:37:29.333090 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:37:29.332835 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-hhh67_openshift-console-operator(2c03b1b7-10b4-43a0-a952-bd191d47aff5)\"" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" podUID="2c03b1b7-10b4-43a0-a952-bd191d47aff5" Jun 08 15:37:30.471374 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.471343 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:30.716573 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.716542 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:30.735915 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.735855 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c"] Jun 08 15:37:30.738006 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.737993 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" Jun 08 15:37:30.740044 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.740026 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zkh9n\"" Jun 08 15:37:30.740118 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.740056 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 08 15:37:30.740118 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.740026 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 08 15:37:30.750798 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.750779 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c"] Jun 08 15:37:30.761681 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.761661 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nf7qm"] Jun 08 15:37:30.763658 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.763643 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.765745 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.765723 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 08 15:37:30.766388 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.766369 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 08 15:37:30.766501 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.766397 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9h8s5\"" Jun 08 15:37:30.766565 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.766531 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 08 15:37:30.805454 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.805428 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:30.805594 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.805489 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-textfile\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.805594 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.805542 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7578954d-074f-4b27-be5c-aa7cf8b194e7-root\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.805594 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.805578 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-tls\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.805799 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.805622 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73e378de-88ad-43cc-8078-ce02081e33d9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-fk89c\" (UID: \"73e378de-88ad-43cc-8078-ce02081e33d9\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" Jun 08 15:37:30.805799 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.805660 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf7tc\" (UniqueName: \"kubernetes.io/projected/73e378de-88ad-43cc-8078-ce02081e33d9-kube-api-access-sf7tc\") pod \"openshift-state-metrics-65f78d5c66-fk89c\" (UID: \"73e378de-88ad-43cc-8078-ce02081e33d9\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" Jun 08 15:37:30.805799 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.805725 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5vqk\" (UniqueName: \"kubernetes.io/projected/7578954d-074f-4b27-be5c-aa7cf8b194e7-kube-api-access-t5vqk\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.805799 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.805756 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73e378de-88ad-43cc-8078-ce02081e33d9-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-fk89c\" (UID: \"73e378de-88ad-43cc-8078-ce02081e33d9\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" Jun 08 15:37:30.805799 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.805791 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.806040 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.805819 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-wtmp\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.806040 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.805854 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7578954d-074f-4b27-be5c-aa7cf8b194e7-sys\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.806040 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.805875 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7578954d-074f-4b27-be5c-aa7cf8b194e7-metrics-client-ca\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.806040 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.805933 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73e378de-88ad-43cc-8078-ce02081e33d9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-fk89c\" (UID: \"73e378de-88ad-43cc-8078-ce02081e33d9\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" Jun 08 15:37:30.806040 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.805980 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-accelerators-collector-config\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.907124 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907101 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7578954d-074f-4b27-be5c-aa7cf8b194e7-root\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.907219 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907127 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-tls\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.907219 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907147 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73e378de-88ad-43cc-8078-ce02081e33d9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-fk89c\" (UID: \"73e378de-88ad-43cc-8078-ce02081e33d9\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" Jun 08 15:37:30.907219 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907165 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf7tc\" (UniqueName: \"kubernetes.io/projected/73e378de-88ad-43cc-8078-ce02081e33d9-kube-api-access-sf7tc\") pod \"openshift-state-metrics-65f78d5c66-fk89c\" (UID: \"73e378de-88ad-43cc-8078-ce02081e33d9\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" Jun 08 15:37:30.907337 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907225 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7578954d-074f-4b27-be5c-aa7cf8b194e7-root\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.907337 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907226 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5vqk\" (UniqueName: \"kubernetes.io/projected/7578954d-074f-4b27-be5c-aa7cf8b194e7-kube-api-access-t5vqk\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.907337 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:37:30.907264 2557 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 08 15:37:30.907337 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907274 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73e378de-88ad-43cc-8078-ce02081e33d9-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-fk89c\" (UID: \"73e378de-88ad-43cc-8078-ce02081e33d9\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" Jun 08 15:37:30.907337 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907306 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.907337 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:37:30.907329 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-tls podName:7578954d-074f-4b27-be5c-aa7cf8b194e7 nodeName:}" failed. No retries permitted until 2026-06-08 15:37:31.407312653 +0000 UTC m=+241.496443680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-tls") pod "node-exporter-nf7qm" (UID: "7578954d-074f-4b27-be5c-aa7cf8b194e7") : secret "node-exporter-tls" not found Jun 08 15:37:30.907711 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907361 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-wtmp\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.907711 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907397 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7578954d-074f-4b27-be5c-aa7cf8b194e7-sys\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.907711 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907419 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7578954d-074f-4b27-be5c-aa7cf8b194e7-metrics-client-ca\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.907711 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907497 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73e378de-88ad-43cc-8078-ce02081e33d9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-fk89c\" (UID: \"73e378de-88ad-43cc-8078-ce02081e33d9\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" Jun 08 15:37:30.907711 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907536 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-accelerators-collector-config\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.907711 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907552 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7578954d-074f-4b27-be5c-aa7cf8b194e7-sys\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.907711 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907583 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-wtmp\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.907711 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907641 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-textfile\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.907711 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:37:30.907672 2557 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 08 15:37:30.908143 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:37:30.907721 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73e378de-88ad-43cc-8078-ce02081e33d9-openshift-state-metrics-tls podName:73e378de-88ad-43cc-8078-ce02081e33d9 nodeName:}" failed. No retries permitted until 2026-06-08 15:37:31.407704943 +0000 UTC m=+241.496835979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/73e378de-88ad-43cc-8078-ce02081e33d9-openshift-state-metrics-tls") pod "openshift-state-metrics-65f78d5c66-fk89c" (UID: "73e378de-88ad-43cc-8078-ce02081e33d9") : secret "openshift-state-metrics-tls" not found Jun 08 15:37:30.908143 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.907983 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-textfile\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.908143 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.908092 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73e378de-88ad-43cc-8078-ce02081e33d9-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-fk89c\" (UID: \"73e378de-88ad-43cc-8078-ce02081e33d9\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" Jun 08 15:37:30.908143 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.908128 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7578954d-074f-4b27-be5c-aa7cf8b194e7-metrics-client-ca\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.908268 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.908186 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-accelerators-collector-config\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.909536 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.909517 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.909604 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.909547 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73e378de-88ad-43cc-8078-ce02081e33d9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-fk89c\" (UID: \"73e378de-88ad-43cc-8078-ce02081e33d9\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" Jun 08 15:37:30.915090 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.915071 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5vqk\" (UniqueName: \"kubernetes.io/projected/7578954d-074f-4b27-be5c-aa7cf8b194e7-kube-api-access-t5vqk\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:30.915261 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.915244 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf7tc\" (UniqueName: \"kubernetes.io/projected/73e378de-88ad-43cc-8078-ce02081e33d9-kube-api-access-sf7tc\") pod \"openshift-state-metrics-65f78d5c66-fk89c\" (UID: \"73e378de-88ad-43cc-8078-ce02081e33d9\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" Jun 08 15:37:30.922365 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:30.922329 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:31.022148 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:31.022044 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:31.126589 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:31.126567 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:31.223652 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:31.223630 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:31.323993 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:31.323912 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:31.411425 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:31.411396 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73e378de-88ad-43cc-8078-ce02081e33d9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-fk89c\" (UID: \"73e378de-88ad-43cc-8078-ce02081e33d9\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" Jun 08 15:37:31.411587 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:31.411567 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-tls\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:31.413791 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:31.413767 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73e378de-88ad-43cc-8078-ce02081e33d9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-fk89c\" (UID: \"73e378de-88ad-43cc-8078-ce02081e33d9\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" Jun 08 15:37:31.413975 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:31.413939 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7578954d-074f-4b27-be5c-aa7cf8b194e7-node-exporter-tls\") pod \"node-exporter-nf7qm\" (UID: \"7578954d-074f-4b27-be5c-aa7cf8b194e7\") " pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:31.646871 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:31.646838 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" Jun 08 15:37:31.672576 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:31.672549 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nf7qm" Jun 08 15:37:31.713622 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:31.713593 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:31.780314 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:31.780284 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c"] Jun 08 15:37:31.783635 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:37:31.783611 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73e378de_88ad_43cc_8078_ce02081e33d9.slice/crio-9d71dbf148728e3269d780e25f5da5da06726c5b4b7ef33b56738d04c5422df5 WatchSource:0}: Error finding container 9d71dbf148728e3269d780e25f5da5da06726c5b4b7ef33b56738d04c5422df5: Status 404 returned error can't find the container with id 9d71dbf148728e3269d780e25f5da5da06726c5b4b7ef33b56738d04c5422df5 Jun 08 15:37:31.785435 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:31.785411 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:32.064206 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.064145 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nf7qm" event={"ID":"7578954d-074f-4b27-be5c-aa7cf8b194e7","Type":"ContainerStarted","Data":"0c84d063adfc0a41edba1db7480a7bf75b3ea80fae9fb9171663ff4ed17f78f1"} Jun 08 15:37:32.066561 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.066531 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" event={"ID":"73e378de-88ad-43cc-8078-ce02081e33d9","Type":"ContainerStarted","Data":"9171748ae3154d30d1eb0e5aa4fccc4e54a039a2b176c693fa08d11154abb9f2"} Jun 08 15:37:32.066702 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.066592 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" event={"ID":"73e378de-88ad-43cc-8078-ce02081e33d9","Type":"ContainerStarted","Data":"f0e83d5ab93f3d70f190e8b772a89fdadef21d3a530a9bf15cb135e227b98700"} Jun 08 15:37:32.066702 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.066621 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" event={"ID":"73e378de-88ad-43cc-8078-ce02081e33d9","Type":"ContainerStarted","Data":"9d71dbf148728e3269d780e25f5da5da06726c5b4b7ef33b56738d04c5422df5"} Jun 08 15:37:32.122947 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.122918 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:32.714168 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.712233 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-57b555fc9f-79kwk"] Jun 08 15:37:32.715207 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.715190 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.717289 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.717260 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 08 15:37:32.717403 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.717311 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 08 15:37:32.717403 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.717357 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 08 15:37:32.717403 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.717361 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 08 15:37:32.717870 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.717853 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-swrxm\"" Jun 08 15:37:32.717870 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.717865 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 08 15:37:32.718033 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.717865 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fldi20p4ojvl9\"" Jun 08 15:37:32.728076 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.728056 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-57b555fc9f-79kwk"] Jun 08 15:37:32.824181 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.824151 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.824322 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.824210 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmph9\" (UniqueName: \"kubernetes.io/projected/742c5a1b-bf10-4705-857b-46c516917bf9-kube-api-access-fmph9\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.824322 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.824285 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.824434 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.824330 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.824434 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.824365 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-grpc-tls\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.824524 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.824432 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.824578 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.824516 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-thanos-querier-tls\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.824578 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.824539 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/742c5a1b-bf10-4705-857b-46c516917bf9-metrics-client-ca\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.925706 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.925668 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-grpc-tls\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.925880 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.925730 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.925880 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.925778 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-thanos-querier-tls\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.925880 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.925804 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/742c5a1b-bf10-4705-857b-46c516917bf9-metrics-client-ca\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.925880 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.925873 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.926060 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.925942 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmph9\" (UniqueName: \"kubernetes.io/projected/742c5a1b-bf10-4705-857b-46c516917bf9-kube-api-access-fmph9\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.926060 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.925995 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.926060 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.926025 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.927102 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.927050 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/742c5a1b-bf10-4705-857b-46c516917bf9-metrics-client-ca\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.928856 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.928829 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.928951 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.928879 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-thanos-querier-tls\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.929246 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.929193 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.929338 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.929277 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.929338 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.929312 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-grpc-tls\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.929659 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.929637 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/742c5a1b-bf10-4705-857b-46c516917bf9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:32.935269 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:32.935244 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmph9\" (UniqueName: \"kubernetes.io/projected/742c5a1b-bf10-4705-857b-46c516917bf9-kube-api-access-fmph9\") pod \"thanos-querier-57b555fc9f-79kwk\" (UID: \"742c5a1b-bf10-4705-857b-46c516917bf9\") " pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:33.024403 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:33.024340 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:33.071850 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:33.071818 2557 generic.go:356] "Generic (PLEG): container finished" podID="7578954d-074f-4b27-be5c-aa7cf8b194e7" containerID="a90155958da55e9c9e929f4e4e385218454bcf847da5cf250fcd8ff2291a5dbf" exitCode=0 Jun 08 15:37:33.071961 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:33.071874 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nf7qm" event={"ID":"7578954d-074f-4b27-be5c-aa7cf8b194e7","Type":"ContainerDied","Data":"a90155958da55e9c9e929f4e4e385218454bcf847da5cf250fcd8ff2291a5dbf"} Jun 08 15:37:33.178677 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:33.178650 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-57b555fc9f-79kwk"] Jun 08 15:37:33.181123 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:37:33.181096 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod742c5a1b_bf10_4705_857b_46c516917bf9.slice/crio-023ae73532e0115635a89480f7997e2697302027929b7539dd3b4dc0a6c6ebf0 WatchSource:0}: Error finding container 023ae73532e0115635a89480f7997e2697302027929b7539dd3b4dc0a6c6ebf0: Status 404 returned error can't find the container with id 023ae73532e0115635a89480f7997e2697302027929b7539dd3b4dc0a6c6ebf0 Jun 08 15:37:33.653474 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:33.653423 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:33.707953 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:33.707927 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:33.871835 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:33.871782 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:34.033752 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:34.033665 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:34.078179 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:34.078147 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" event={"ID":"73e378de-88ad-43cc-8078-ce02081e33d9","Type":"ContainerStarted","Data":"62811f6b1dc6fde4a0cf6b543fe228fc3c39da41d9da596d08c45a1a3258c22b"} Jun 08 15:37:34.080670 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:34.080646 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nf7qm" event={"ID":"7578954d-074f-4b27-be5c-aa7cf8b194e7","Type":"ContainerStarted","Data":"dc407e96262f8bdd612eb03da17de206fb75743d0379fa59215da6631072a6ac"} Jun 08 15:37:34.080819 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:34.080690 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nf7qm" event={"ID":"7578954d-074f-4b27-be5c-aa7cf8b194e7","Type":"ContainerStarted","Data":"42d33a04ed7281c4a7691979859757fbc2ebf92bfab6d5407a1b018f22c4c91d"} Jun 08 15:37:34.082074 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:34.082048 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" event={"ID":"742c5a1b-bf10-4705-857b-46c516917bf9","Type":"ContainerStarted","Data":"023ae73532e0115635a89480f7997e2697302027929b7539dd3b4dc0a6c6ebf0"} Jun 08 15:37:34.094353 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:34.094301 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-fk89c" podStartSLOduration=2.97431191 podStartE2EDuration="4.094280027s" totalImagesPullingTime="1.119968117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:37:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:37:34.093671573 +0000 UTC m=+244.182802622" watchObservedRunningTime="2026-06-08 15:37:34.094280027 +0000 UTC m=+244.183411080" Jun 08 15:37:34.111578 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:34.110935 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nf7qm" podStartSLOduration=3.268939907 podStartE2EDuration="4.110919274s" totalImagesPullingTime="769.052217ms" totalInitContainerRuntime="72.92715ms" isStatefulPod=true podCreationTimestamp="2026-06-08 15:37:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:37:34.109977413 +0000 UTC m=+244.199108484" watchObservedRunningTime="2026-06-08 15:37:34.110919274 +0000 UTC m=+244.200050328" Jun 08 15:37:35.481072 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:35.480995 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:35.484072 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:35.484047 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-ldc69"] Jun 08 15:37:35.486484 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:35.486454 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-ldc69" Jun 08 15:37:35.488621 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:35.488598 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 08 15:37:35.488715 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:35.488631 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-fg9sd\"" Jun 08 15:37:35.494923 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:35.494866 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-ldc69"] Jun 08 15:37:35.551800 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:35.551765 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/18622679-3a6c-45f7-adb0-7a2f922968c4-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-ldc69\" (UID: \"18622679-3a6c-45f7-adb0-7a2f922968c4\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-ldc69" Jun 08 15:37:35.551941 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:35.551838 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v469t\" (UniqueName: \"kubernetes.io/projected/18622679-3a6c-45f7-adb0-7a2f922968c4-kube-api-access-v469t\") pod \"monitoring-plugin-76b68cc874-ldc69\" (UID: \"18622679-3a6c-45f7-adb0-7a2f922968c4\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-ldc69" Jun 08 15:37:35.653113 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:35.653085 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/18622679-3a6c-45f7-adb0-7a2f922968c4-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-ldc69\" (UID: \"18622679-3a6c-45f7-adb0-7a2f922968c4\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-ldc69" Jun 08 15:37:35.653254 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:35.653140 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v469t\" (UniqueName: \"kubernetes.io/projected/18622679-3a6c-45f7-adb0-7a2f922968c4-kube-api-access-v469t\") pod \"monitoring-plugin-76b68cc874-ldc69\" (UID: \"18622679-3a6c-45f7-adb0-7a2f922968c4\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-ldc69" Jun 08 15:37:35.653254 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:37:35.653225 2557 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 08 15:37:35.653336 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:37:35.653290 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18622679-3a6c-45f7-adb0-7a2f922968c4-monitoring-plugin-cert podName:18622679-3a6c-45f7-adb0-7a2f922968c4 nodeName:}" failed. No retries permitted until 2026-06-08 15:37:36.153274533 +0000 UTC m=+246.242405561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/18622679-3a6c-45f7-adb0-7a2f922968c4-monitoring-plugin-cert") pod "monitoring-plugin-76b68cc874-ldc69" (UID: "18622679-3a6c-45f7-adb0-7a2f922968c4") : secret "monitoring-plugin-cert" not found Jun 08 15:37:35.661377 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:35.661348 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v469t\" (UniqueName: \"kubernetes.io/projected/18622679-3a6c-45f7-adb0-7a2f922968c4-kube-api-access-v469t\") pod \"monitoring-plugin-76b68cc874-ldc69\" (UID: \"18622679-3a6c-45f7-adb0-7a2f922968c4\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-ldc69" Jun 08 15:37:35.703165 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:35.703137 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:36.093877 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.093852 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" event={"ID":"742c5a1b-bf10-4705-857b-46c516917bf9","Type":"ContainerStarted","Data":"8d58c2526fde58c8285f60480bf94b03e83bcfaa9d0886c1ca9aa4316a66f6fe"} Jun 08 15:37:36.093964 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.093911 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" event={"ID":"742c5a1b-bf10-4705-857b-46c516917bf9","Type":"ContainerStarted","Data":"30df5be11d60a47481922557b308456c1968b933f6dc4d9e0657e2119df095f6"} Jun 08 15:37:36.093964 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.093939 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" event={"ID":"742c5a1b-bf10-4705-857b-46c516917bf9","Type":"ContainerStarted","Data":"28f5e2f6852ffa8d782b32d8e3b31764a17208b5a6fba3716d2c4d31600c7eb1"} Jun 08 15:37:36.094034 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.093964 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" event={"ID":"742c5a1b-bf10-4705-857b-46c516917bf9","Type":"ContainerStarted","Data":"6981e013fb9480001304dd1e761ee799666e9c11b34de2be50e3c31ba06fa8d6"} Jun 08 15:37:36.158943 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.158917 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/18622679-3a6c-45f7-adb0-7a2f922968c4-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-ldc69\" (UID: \"18622679-3a6c-45f7-adb0-7a2f922968c4\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-ldc69" Jun 08 15:37:36.161297 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.161274 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/18622679-3a6c-45f7-adb0-7a2f922968c4-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-ldc69\" (UID: \"18622679-3a6c-45f7-adb0-7a2f922968c4\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-ldc69" Jun 08 15:37:36.396108 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.396086 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-ldc69" Jun 08 15:37:36.515272 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.515243 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-ldc69"] Jun 08 15:37:36.518092 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:37:36.518065 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18622679_3a6c_45f7_adb0_7a2f922968c4.slice/crio-83429c25e9f9b67789c0e6352b276060900abaaa3c68d563118b201a2a3f9156 WatchSource:0}: Error finding container 83429c25e9f9b67789c0e6352b276060900abaaa3c68d563118b201a2a3f9156: Status 404 returned error can't find the container with id 83429c25e9f9b67789c0e6352b276060900abaaa3c68d563118b201a2a3f9156 Jun 08 15:37:36.884379 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.884340 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:36.891903 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.891877 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 15:37:36.895669 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.895647 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.899211 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.899191 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 08 15:37:36.899926 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.899587 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 08 15:37:36.899926 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.899664 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 08 15:37:36.899926 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.899711 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 08 15:37:36.899926 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.899732 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 08 15:37:36.899926 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.899834 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 08 15:37:36.899926 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.899845 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 08 15:37:36.900354 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.900135 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 08 15:37:36.900354 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.900167 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 08 15:37:36.900354 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.900227 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 08 15:37:36.900527 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.900505 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d3nt4ki9tvnek\"" Jun 08 15:37:36.901015 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.900814 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 08 15:37:36.901015 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.900831 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 08 15:37:36.901015 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.900866 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 08 15:37:36.901015 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.900918 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 08 15:37:36.901413 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.901397 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v9qwf\"" Jun 08 15:37:36.902103 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.902080 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 08 15:37:36.910985 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.910899 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 15:37:36.966296 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966266 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.966456 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966302 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-config\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.966456 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966321 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.966456 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966337 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn68s\" (UniqueName: \"kubernetes.io/projected/a01f0b27-4507-412a-a5b3-76adda68a09d-kube-api-access-cn68s\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.966456 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966411 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.966639 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966486 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.966639 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966520 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.966639 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966545 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.966639 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966561 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.966639 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966622 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-web-config\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.966863 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966655 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.966863 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966690 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.966863 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966723 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a01f0b27-4507-412a-a5b3-76adda68a09d-config-out\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.966863 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966773 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.966863 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966810 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.966863 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966843 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.967109 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966882 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a01f0b27-4507-412a-a5b3-76adda68a09d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.967109 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966906 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.967109 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966956 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:36.967109 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:36.966990 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.067600 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.067568 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.067600 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.067602 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.067856 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.067621 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.067856 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.067644 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a01f0b27-4507-412a-a5b3-76adda68a09d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.067856 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.067776 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.067856 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.067844 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.068060 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.067886 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.068060 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.067930 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.068060 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.067958 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-config\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.068060 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.067987 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.068060 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.068015 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn68s\" (UniqueName: \"kubernetes.io/projected/a01f0b27-4507-412a-a5b3-76adda68a09d-kube-api-access-cn68s\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.068060 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.068059 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.068343 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.068093 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.068343 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.068135 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.068343 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.068178 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.068343 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.068207 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.068343 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.068257 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-web-config\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.068343 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.068296 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.068656 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.068352 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.068656 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.068397 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a01f0b27-4507-412a-a5b3-76adda68a09d-config-out\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.068656 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.068622 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.068805 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.068729 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.069175 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.069146 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.070099 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.068256 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.070595 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.070567 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.071538 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.071490 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.072355 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.072282 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.073411 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.072804 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a01f0b27-4507-412a-a5b3-76adda68a09d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.073411 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.073024 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.073411 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.073160 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.073411 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.073358 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a01f0b27-4507-412a-a5b3-76adda68a09d-config-out\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.073659 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.073591 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.073659 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.073631 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-config\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.075427 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.075403 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.075588 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.075542 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.075759 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.075729 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.075973 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.075951 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.076043 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.076014 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.076567 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.076549 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-web-config\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.078255 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.078229 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn68s\" (UniqueName: \"kubernetes.io/projected/a01f0b27-4507-412a-a5b3-76adda68a09d-kube-api-access-cn68s\") pod \"prometheus-k8s-0\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.102773 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.102742 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" event={"ID":"742c5a1b-bf10-4705-857b-46c516917bf9","Type":"ContainerStarted","Data":"07b29e0296b3c65c1e21ef1641c97e16f292f5dff1b5abb9c1fcc65d4d51fc3d"} Jun 08 15:37:37.102904 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.102815 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" event={"ID":"742c5a1b-bf10-4705-857b-46c516917bf9","Type":"ContainerStarted","Data":"d25960b38eef5860961b53a31ea03057e12087acb4981875042c4369bf9f53f3"} Jun 08 15:37:37.102959 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.102917 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:37.104493 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.104420 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-ldc69" event={"ID":"18622679-3a6c-45f7-adb0-7a2f922968c4","Type":"ContainerStarted","Data":"83429c25e9f9b67789c0e6352b276060900abaaa3c68d563118b201a2a3f9156"} Jun 08 15:37:37.126929 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.126869 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" podStartSLOduration=2.47510043 podStartE2EDuration="5.126848595s" totalImagesPullingTime="2.651748165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:37:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:37:37.124495865 +0000 UTC m=+247.213626928" watchObservedRunningTime="2026-06-08 15:37:37.126848595 +0000 UTC m=+247.215979652" Jun 08 15:37:37.208710 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.208587 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:37.357085 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:37.357054 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 15:37:37.360966 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:37:37.360934 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda01f0b27_4507_412a_a5b3_76adda68a09d.slice/crio-378c2fd8c1e90321fcc3b837ab375ffb8955b50ddce6091da6fb43ec8d0ee287 WatchSource:0}: Error finding container 378c2fd8c1e90321fcc3b837ab375ffb8955b50ddce6091da6fb43ec8d0ee287: Status 404 returned error can't find the container with id 378c2fd8c1e90321fcc3b837ab375ffb8955b50ddce6091da6fb43ec8d0ee287 Jun 08 15:37:38.109976 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:38.109932 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a01f0b27-4507-412a-a5b3-76adda68a09d","Type":"ContainerStarted","Data":"378c2fd8c1e90321fcc3b837ab375ffb8955b50ddce6091da6fb43ec8d0ee287"} Jun 08 15:37:39.115123 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:39.115085 2557 generic.go:356] "Generic (PLEG): container finished" podID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerID="5b5a817f60c7e3dfa33f4917675484ac6ac6d51eb098529d5cc246bae934cbd1" exitCode=0 Jun 08 15:37:39.115547 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:39.115175 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a01f0b27-4507-412a-a5b3-76adda68a09d","Type":"ContainerDied","Data":"5b5a817f60c7e3dfa33f4917675484ac6ac6d51eb098529d5cc246bae934cbd1"} Jun 08 15:37:39.116702 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:39.116668 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-ldc69" event={"ID":"18622679-3a6c-45f7-adb0-7a2f922968c4","Type":"ContainerStarted","Data":"8751efcb890e19772397d73c356ee9abd4bebb2d17ab827e69028799f7a36bbc"} Jun 08 15:37:39.116815 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:39.116719 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-ldc69" Jun 08 15:37:39.121481 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:39.121450 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-ldc69" Jun 08 15:37:39.153285 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:39.153244 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-76b68cc874-ldc69" podStartSLOduration=2.175934628 podStartE2EDuration="4.153223856s" totalImagesPullingTime="1.977289228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:37:39.151928415 +0000 UTC m=+249.241059464" watchObservedRunningTime="2026-06-08 15:37:39.153223856 +0000 UTC m=+249.242354922" Jun 08 15:37:42.121584 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:42.121547 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs\") pod \"network-metrics-daemon-hct6z\" (UID: \"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58\") " pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:37:42.124344 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:42.124312 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58-metrics-certs\") pod \"network-metrics-daemon-hct6z\" (UID: \"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58\") " pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:37:42.135788 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:42.135764 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-54ltp\"" Jun 08 15:37:42.144357 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:42.144331 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hct6z" Jun 08 15:37:42.327538 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:42.327511 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hct6z"] Jun 08 15:37:42.334555 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:37:42.334527 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75dc79a6_6f73_4df4_a4df_a7b6f3eb5f58.slice/crio-b0e11af254955d4273a0d5c83dc66f6f8e0a7d3f197c3bdad245ca5b06c47d22 WatchSource:0}: Error finding container b0e11af254955d4273a0d5c83dc66f6f8e0a7d3f197c3bdad245ca5b06c47d22: Status 404 returned error can't find the container with id b0e11af254955d4273a0d5c83dc66f6f8e0a7d3f197c3bdad245ca5b06c47d22 Jun 08 15:37:43.117285 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:43.117256 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-57b555fc9f-79kwk" Jun 08 15:37:43.141100 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:43.141059 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a01f0b27-4507-412a-a5b3-76adda68a09d","Type":"ContainerStarted","Data":"d758da6bd3932d97919c3455742c1aab6f342abbc3312c802785109433eb1ca2"} Jun 08 15:37:43.141608 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:43.141150 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a01f0b27-4507-412a-a5b3-76adda68a09d","Type":"ContainerStarted","Data":"e9cde7715a8d3f606631753c14e2620c2ce732da76f80b73a6a8a8391db62f6f"} Jun 08 15:37:43.141608 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:43.141193 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a01f0b27-4507-412a-a5b3-76adda68a09d","Type":"ContainerStarted","Data":"eecfedbdf30efb8006520c4f211bcad8510d713d399c69ded931e3a896d1bb9f"} Jun 08 15:37:43.141608 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:43.141240 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a01f0b27-4507-412a-a5b3-76adda68a09d","Type":"ContainerStarted","Data":"d1c30ec61f54ef8deb4d031e610aba759a66dc259c95ba94b85ffa7a1cc3892d"} Jun 08 15:37:43.141608 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:43.141280 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a01f0b27-4507-412a-a5b3-76adda68a09d","Type":"ContainerStarted","Data":"bae926091380ea4def07a2218dca672ba10c8c039f41230a8d89ef16743d1fae"} Jun 08 15:37:43.141608 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:43.141321 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a01f0b27-4507-412a-a5b3-76adda68a09d","Type":"ContainerStarted","Data":"b842fa9e62d6f3eca2fb6f6ee178ceb9fc1e2c897f3c94e9682d67468b35149b"} Jun 08 15:37:43.142633 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:43.142608 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hct6z" event={"ID":"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58","Type":"ContainerStarted","Data":"b0e11af254955d4273a0d5c83dc66f6f8e0a7d3f197c3bdad245ca5b06c47d22"} Jun 08 15:37:43.167971 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:43.167917 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.90097119 podStartE2EDuration="7.167899259s" totalImagesPullingTime="4.240624933s" totalInitContainerRuntime="26.303136ms" isStatefulPod=false podCreationTimestamp="2026-06-08 15:37:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:37:43.164940165 +0000 UTC m=+253.254071216" watchObservedRunningTime="2026-06-08 15:37:43.167899259 +0000 UTC m=+253.257030312" Jun 08 15:37:43.332658 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:43.332624 2557 scope.go:117] "RemoveContainer" containerID="21513c1ab3e8546ea50f4d4eb0243e7007a460a717076e125a0010d540855ba4" Jun 08 15:37:44.003530 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.003432 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:44.098385 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.098343 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:44.148781 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.148749 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hct6z" event={"ID":"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58","Type":"ContainerStarted","Data":"c1aef86050a86e5ae676f1d75279a02e6a60cea708c64ec910ae3f53f6f78b5b"} Jun 08 15:37:44.149212 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.148808 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hct6z" event={"ID":"75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58","Type":"ContainerStarted","Data":"54d9ac781c20b450df92bf00f3e4cafb84fb90e69ff7168092af0c97ef8838fb"} Jun 08 15:37:44.150922 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.150894 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 15:37:44.151218 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.151167 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" event={"ID":"2c03b1b7-10b4-43a0-a952-bd191d47aff5","Type":"ContainerStarted","Data":"e9757511cfb27e205472c1e3144d836e35f4f6075b2741652c67f92a645867c9"} Jun 08 15:37:44.152225 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.152197 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:37:44.156650 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.156632 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" Jun 08 15:37:44.165562 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.165520 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hct6z" podStartSLOduration=253.231811643 podStartE2EDuration="4m14.165503709s" totalImagesPullingTime="933.692066ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:37:44.164308322 +0000 UTC m=+254.253439373" watchObservedRunningTime="2026-06-08 15:37:44.165503709 +0000 UTC m=+254.254634761" Jun 08 15:37:44.169289 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.169262 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:44.185377 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.185325 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-668fdbf85c-hhh67" podStartSLOduration=53.61512052 podStartE2EDuration="56.185307083s" totalImagesPullingTime="2.570186563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:36:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:37:44.184137954 +0000 UTC m=+254.273269005" watchObservedRunningTime="2026-06-08 15:37:44.185307083 +0000 UTC m=+254.274438133" Jun 08 15:37:44.365476 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.365376 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:44.426979 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.426943 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:44.480435 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.480407 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:44.649377 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.649349 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:44.749741 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.749700 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:44.859796 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:44.859758 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:45.051472 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:45.051382 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:45.151690 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:45.151661 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:45.258331 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:45.258303 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:45.314894 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:45.314804 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:45.665252 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:45.665208 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:46.070050 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:46.069958 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:46.182527 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:46.182494 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:47.209719 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:47.209686 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:37:47.277808 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:47.277774 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:52.108297 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:52.108259 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:52.160534 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:52.160504 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:59.884404 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:59.884372 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:37:59.935352 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:37:59.935317 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:02.294478 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:02.294430 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:02.344627 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:02.344583 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:02.562573 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:02.562449 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:02.663337 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:02.663305 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:03.690306 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:03.690253 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:05.042254 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:05.042228 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l8rfj_9865e636-5527-48fe-b4a8-de43e6b507ec/dns-node-resolver/0.log" Jun 08 15:38:05.842169 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:05.842140 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5t7fb_d777d945-3cc7-4e04-b64b-f021f129aea7/node-ca/0.log" Jun 08 15:38:06.242633 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:06.242606 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-664d9c4559-shg98_ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50/router/0.log" Jun 08 15:38:06.641882 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:06.641854 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mvlgf_ac6d6aa0-340a-4156-aa2d-60444b6f4fcb/serve-healthcheck-canary/0.log" Jun 08 15:38:07.571497 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:07.571419 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:07.625752 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:07.625716 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:09.887867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:09.887814 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:09.937806 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:09.937779 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:12.141927 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:12.141893 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:12.890998 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:12.890944 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:12.946794 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:12.946761 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:13.287039 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:13.286955 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:13.347242 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:13.347199 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:14.703750 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:14.703717 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:38:14.706277 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:14.706255 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls\") pod \"image-registry-84f8f6f6b6-jx5jx\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:38:14.804734 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:14.804691 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:38:14.806946 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:14.806917 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9492b29-8d64-43ca-9f70-250f32707cfd-metrics-tls\") pod \"dns-default-t8nc2\" (UID: \"b9492b29-8d64-43ca-9f70-250f32707cfd\") " pod="openshift-dns/dns-default-t8nc2" Jun 08 15:38:14.836995 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:14.836965 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p7nwf\"" Jun 08 15:38:14.837493 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:14.837453 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mdbb6\"" Jun 08 15:38:14.844475 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:14.844435 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t8nc2" Jun 08 15:38:14.844581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:14.844489 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:38:15.049492 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:15.049273 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t8nc2"] Jun 08 15:38:15.053149 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:38:15.053117 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9492b29_8d64_43ca_9f70_250f32707cfd.slice/crio-8b3a9ef2cebfd39d4686ba972e742dad83004ec55ba1562f30c3abc0fe152709 WatchSource:0}: Error finding container 8b3a9ef2cebfd39d4686ba972e742dad83004ec55ba1562f30c3abc0fe152709: Status 404 returned error can't find the container with id 8b3a9ef2cebfd39d4686ba972e742dad83004ec55ba1562f30c3abc0fe152709 Jun 08 15:38:15.083094 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:15.083060 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84f8f6f6b6-jx5jx"] Jun 08 15:38:15.087884 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:38:15.087857 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a2ada9_1340_40f3_b339_f2a682aecf8d.slice/crio-5cf93dfac92b04aa9de12bd148ea5372d2c97e1daeb1c2aee09532c64f6ffcbb WatchSource:0}: Error finding container 5cf93dfac92b04aa9de12bd148ea5372d2c97e1daeb1c2aee09532c64f6ffcbb: Status 404 returned error can't find the container with id 5cf93dfac92b04aa9de12bd148ea5372d2c97e1daeb1c2aee09532c64f6ffcbb Jun 08 15:38:15.280326 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:15.279744 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" event={"ID":"a0a2ada9-1340-40f3-b339-f2a682aecf8d","Type":"ContainerStarted","Data":"0d15f9a4c709e94b86806309a82a02443fbdde835e850a971404bc3d0f8a9fa9"} Jun 08 15:38:15.280326 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:15.279804 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" event={"ID":"a0a2ada9-1340-40f3-b339-f2a682aecf8d","Type":"ContainerStarted","Data":"5cf93dfac92b04aa9de12bd148ea5372d2c97e1daeb1c2aee09532c64f6ffcbb"} Jun 08 15:38:15.280326 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:15.279847 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:38:15.281516 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:15.281491 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t8nc2" event={"ID":"b9492b29-8d64-43ca-9f70-250f32707cfd","Type":"ContainerStarted","Data":"8b3a9ef2cebfd39d4686ba972e742dad83004ec55ba1562f30c3abc0fe152709"} Jun 08 15:38:15.297848 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:15.297800 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" podStartSLOduration=285.297781111 podStartE2EDuration="4m45.297781111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:33:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:38:15.29676238 +0000 UTC m=+285.385893424" watchObservedRunningTime="2026-06-08 15:38:15.297781111 +0000 UTC m=+285.386912161" Jun 08 15:38:17.300734 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:17.297912 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t8nc2" event={"ID":"b9492b29-8d64-43ca-9f70-250f32707cfd","Type":"ContainerStarted","Data":"aa0067f6af7dc9d8951544aafa9aa84df6898f29ceacca4d838d93c807a5bfe6"} Jun 08 15:38:18.303096 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:18.303061 2557 generic.go:356] "Generic (PLEG): container finished" podID="853db474-a505-43a0-8f8d-fb060c8bcc70" containerID="096bff9a3ecec7d1d71de5bfec4fdec6ec2f966e0b87f336823df47a6a35f5d2" exitCode=0 Jun 08 15:38:18.303546 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:18.303138 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp" event={"ID":"853db474-a505-43a0-8f8d-fb060c8bcc70","Type":"ContainerDied","Data":"096bff9a3ecec7d1d71de5bfec4fdec6ec2f966e0b87f336823df47a6a35f5d2"} Jun 08 15:38:18.303629 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:18.303555 2557 scope.go:117] "RemoveContainer" containerID="096bff9a3ecec7d1d71de5bfec4fdec6ec2f966e0b87f336823df47a6a35f5d2" Jun 08 15:38:18.304987 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:18.304968 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t8nc2" event={"ID":"b9492b29-8d64-43ca-9f70-250f32707cfd","Type":"ContainerStarted","Data":"0e4be02c520542dd392ec98e6e73e9d3f04fbec5dcddb62c7ad8473f855a7ed9"} Jun 08 15:38:18.305144 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:18.305124 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-t8nc2" Jun 08 15:38:18.335235 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:18.334699 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-t8nc2" podStartSLOduration=252.319994418 podStartE2EDuration="4m14.334680721s" totalImagesPullingTime="2.014686303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:38:18.333496965 +0000 UTC m=+288.422628016" watchObservedRunningTime="2026-06-08 15:38:18.334680721 +0000 UTC m=+288.423811773" Jun 08 15:38:19.310315 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:19.310276 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-9pkwp" event={"ID":"853db474-a505-43a0-8f8d-fb060c8bcc70","Type":"ContainerStarted","Data":"abffa27c3929fa7c89f051f159932d859da0323828079e9ac4d5431d6c20988a"} Jun 08 15:38:22.430942 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:22.430907 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:22.480211 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:22.480180 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:22.958140 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:22.958108 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:28.312931 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:28.312903 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-t8nc2" Jun 08 15:38:29.357148 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:29.357066 2557 generic.go:356] "Generic (PLEG): container finished" podID="bb4ff32f-3b7d-4b34-9637-826937bd8b4a" containerID="f196b5ba4e03a1c91107d7876ec099e69472823f79c702c90260f6523285fbaf" exitCode=1 Jun 08 15:38:29.357607 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:29.357139 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" event={"ID":"bb4ff32f-3b7d-4b34-9637-826937bd8b4a","Type":"ContainerDied","Data":"f196b5ba4e03a1c91107d7876ec099e69472823f79c702c90260f6523285fbaf"} Jun 08 15:38:29.357607 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:29.357194 2557 scope.go:117] "RemoveContainer" containerID="6c21fdc5686867b5b510736bf8732043101d1cd540762d8968b53f1e7f6121f2" Jun 08 15:38:29.357607 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:29.357486 2557 scope.go:117] "RemoveContainer" containerID="f196b5ba4e03a1c91107d7876ec099e69472823f79c702c90260f6523285fbaf" Jun 08 15:38:29.357731 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:38:29.357669 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-77776c944d-f6bd4_open-cluster-management-agent-addon(bb4ff32f-3b7d-4b34-9637-826937bd8b4a)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" podUID="bb4ff32f-3b7d-4b34-9637-826937bd8b4a" Jun 08 15:38:29.852885 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:29.852856 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:38:30.280123 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:30.280095 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 15:38:30.286593 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:30.286562 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 15:38:30.289043 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:30.289024 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 15:38:30.292091 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:30.292073 2557 kubelet.go:1648] "Image garbage collection succeeded" Jun 08 15:38:30.295015 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:30.294996 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 15:38:30.362030 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:30.361918 2557 scope.go:117] "RemoveContainer" containerID="f196b5ba4e03a1c91107d7876ec099e69472823f79c702c90260f6523285fbaf" Jun 08 15:38:30.370138 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:38:30.362075 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-77776c944d-f6bd4_open-cluster-management-agent-addon(bb4ff32f-3b7d-4b34-9637-826937bd8b4a)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" podUID="bb4ff32f-3b7d-4b34-9637-826937bd8b4a" Jun 08 15:38:32.050398 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:32.050352 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:33.775808 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:33.775738 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:38:33.776168 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:33.776134 2557 scope.go:117] "RemoveContainer" containerID="f196b5ba4e03a1c91107d7876ec099e69472823f79c702c90260f6523285fbaf" Jun 08 15:38:33.776306 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:38:33.776284 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-77776c944d-f6bd4_open-cluster-management-agent-addon(bb4ff32f-3b7d-4b34-9637-826937bd8b4a)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" podUID="bb4ff32f-3b7d-4b34-9637-826937bd8b4a" Jun 08 15:38:34.848851 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:34.848815 2557 patch_prober.go:28] interesting pod/image-registry-84f8f6f6b6-jx5jx container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 15:38:34.849215 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:34.848875 2557 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" podUID="a0a2ada9-1340-40f3-b339-f2a682aecf8d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 15:38:36.292693 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:36.292665 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:38:36.475906 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:36.475865 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:36.525813 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:36.525788 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:37.209548 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:37.209490 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:37.225244 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:37.225220 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:37.405814 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:37.405783 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:37.428514 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:37.428486 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:37.528933 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:37.528851 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:38.893510 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:38.893473 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:38.994313 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:38.994278 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:39.044746 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:39.044720 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:39.208625 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:39.208537 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:39.265974 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:39.265944 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:39.322325 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:39.322297 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:42.632381 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:42.632328 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:42.731156 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:42.731120 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:43.631769 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:43.631735 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84f8f6f6b6-jx5jx"] Jun 08 15:38:43.722319 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:43.722279 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:43.822227 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:43.822185 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:43.923902 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:43.923790 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:49.332643 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:49.332613 2557 scope.go:117] "RemoveContainer" containerID="f196b5ba4e03a1c91107d7876ec099e69472823f79c702c90260f6523285fbaf" Jun 08 15:38:49.437887 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:49.437858 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" event={"ID":"bb4ff32f-3b7d-4b34-9637-826937bd8b4a","Type":"ContainerStarted","Data":"acadd2a1481fc16c55992097713441c163cee48ce958153673a33bb1c4e5603e"} Jun 08 15:38:49.438291 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:49.438235 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:38:49.439920 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:49.439902 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77776c944d-f6bd4" Jun 08 15:38:49.683407 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:49.683368 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:49.784165 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:49.784135 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:49.888158 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:49.888120 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:50.185172 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:50.185134 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:54.721619 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:54.721581 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:54.821835 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:54.821801 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:55.252446 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.252411 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 15:38:55.252751 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.252720 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="prometheus" containerID="cri-o://b842fa9e62d6f3eca2fb6f6ee178ceb9fc1e2c897f3c94e9682d67468b35149b" gracePeriod=600 Jun 08 15:38:55.252751 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.252737 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d758da6bd3932d97919c3455742c1aab6f342abbc3312c802785109433eb1ca2" gracePeriod=600 Jun 08 15:38:55.252932 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.252763 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="thanos-sidecar" containerID="cri-o://d1c30ec61f54ef8deb4d031e610aba759a66dc259c95ba94b85ffa7a1cc3892d" gracePeriod=600 Jun 08 15:38:55.252932 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.252792 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="config-reloader" containerID="cri-o://bae926091380ea4def07a2218dca672ba10c8c039f41230a8d89ef16743d1fae" gracePeriod=600 Jun 08 15:38:55.252932 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.252789 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="kube-rbac-proxy" containerID="cri-o://e9cde7715a8d3f606631753c14e2620c2ce732da76f80b73a6a8a8391db62f6f" gracePeriod=600 Jun 08 15:38:55.252932 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.252740 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="kube-rbac-proxy-web" containerID="cri-o://eecfedbdf30efb8006520c4f211bcad8510d713d399c69ded931e3a896d1bb9f" gracePeriod=600 Jun 08 15:38:55.467599 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.467565 2557 generic.go:356] "Generic (PLEG): container finished" podID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerID="d758da6bd3932d97919c3455742c1aab6f342abbc3312c802785109433eb1ca2" exitCode=0 Jun 08 15:38:55.467599 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.467595 2557 generic.go:356] "Generic (PLEG): container finished" podID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerID="e9cde7715a8d3f606631753c14e2620c2ce732da76f80b73a6a8a8391db62f6f" exitCode=0 Jun 08 15:38:55.467801 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.467605 2557 generic.go:356] "Generic (PLEG): container finished" podID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerID="eecfedbdf30efb8006520c4f211bcad8510d713d399c69ded931e3a896d1bb9f" exitCode=0 Jun 08 15:38:55.467801 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.467616 2557 generic.go:356] "Generic (PLEG): container finished" podID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerID="d1c30ec61f54ef8deb4d031e610aba759a66dc259c95ba94b85ffa7a1cc3892d" exitCode=0 Jun 08 15:38:55.467801 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.467626 2557 generic.go:356] "Generic (PLEG): container finished" podID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerID="bae926091380ea4def07a2218dca672ba10c8c039f41230a8d89ef16743d1fae" exitCode=0 Jun 08 15:38:55.467801 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.467635 2557 generic.go:356] "Generic (PLEG): container finished" podID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerID="b842fa9e62d6f3eca2fb6f6ee178ceb9fc1e2c897f3c94e9682d67468b35149b" exitCode=0 Jun 08 15:38:55.467801 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.467643 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a01f0b27-4507-412a-a5b3-76adda68a09d","Type":"ContainerDied","Data":"d758da6bd3932d97919c3455742c1aab6f342abbc3312c802785109433eb1ca2"} Jun 08 15:38:55.467801 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.467741 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a01f0b27-4507-412a-a5b3-76adda68a09d","Type":"ContainerDied","Data":"e9cde7715a8d3f606631753c14e2620c2ce732da76f80b73a6a8a8391db62f6f"} Jun 08 15:38:55.467801 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.467788 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a01f0b27-4507-412a-a5b3-76adda68a09d","Type":"ContainerDied","Data":"eecfedbdf30efb8006520c4f211bcad8510d713d399c69ded931e3a896d1bb9f"} Jun 08 15:38:55.468044 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.467831 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a01f0b27-4507-412a-a5b3-76adda68a09d","Type":"ContainerDied","Data":"d1c30ec61f54ef8deb4d031e610aba759a66dc259c95ba94b85ffa7a1cc3892d"} Jun 08 15:38:55.468044 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.467873 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a01f0b27-4507-412a-a5b3-76adda68a09d","Type":"ContainerDied","Data":"bae926091380ea4def07a2218dca672ba10c8c039f41230a8d89ef16743d1fae"} Jun 08 15:38:55.468044 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.467914 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a01f0b27-4507-412a-a5b3-76adda68a09d","Type":"ContainerDied","Data":"b842fa9e62d6f3eca2fb6f6ee178ceb9fc1e2c897f3c94e9682d67468b35149b"} Jun 08 15:38:55.511307 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.511251 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:55.665296 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665251 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-0\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665509 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665307 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665509 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665336 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-tls\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665509 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665362 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-kube-rbac-proxy\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665509 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665385 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a01f0b27-4507-412a-a5b3-76adda68a09d-tls-assets\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665509 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665409 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-config\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665509 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665430 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-web-config\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665509 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665490 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-serving-certs-ca-bundle\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665881 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665521 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-db\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665881 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665549 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-metrics-client-ca\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665881 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665572 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a01f0b27-4507-412a-a5b3-76adda68a09d-config-out\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665881 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665600 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-1\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665881 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665624 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-grpc-tls\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665881 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665664 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-2\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665881 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665703 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-kubelet-serving-ca-bundle\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665881 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665738 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-thanos-prometheus-http-client-file\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665881 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665824 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-metrics-client-certs\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.665881 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665854 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.666342 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665884 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn68s\" (UniqueName: \"kubernetes.io/projected/a01f0b27-4507-412a-a5b3-76adda68a09d-kube-api-access-cn68s\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.666342 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.665910 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-trusted-ca-bundle\") pod \"a01f0b27-4507-412a-a5b3-76adda68a09d\" (UID: \"a01f0b27-4507-412a-a5b3-76adda68a09d\") " Jun 08 15:38:55.666444 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.666392 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 15:38:55.666444 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.666426 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 15:38:55.666642 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.666580 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 15:38:55.667721 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.667004 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 15:38:55.667721 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.667152 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 15:38:55.667721 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.667545 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 15:38:55.667721 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.667685 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 15:38:55.668229 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.668164 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 15:38:55.668595 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.668567 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a01f0b27-4507-412a-a5b3-76adda68a09d-config-out" (OuterVolumeSpecName: "config-out") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 15:38:55.668972 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.668947 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 15:38:55.669422 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.669366 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 15:38:55.669786 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.669755 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-config" (OuterVolumeSpecName: "config") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 15:38:55.669885 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.669817 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 15:38:55.669957 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.669901 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a01f0b27-4507-412a-a5b3-76adda68a09d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 15:38:55.670252 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.670225 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 15:38:55.671097 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.671068 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 15:38:55.671508 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.671485 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 15:38:55.671939 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.671912 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a01f0b27-4507-412a-a5b3-76adda68a09d-kube-api-access-cn68s" (OuterVolumeSpecName: "kube-api-access-cn68s") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "kube-api-access-cn68s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 15:38:55.672023 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.671998 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 15:38:55.679891 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.679871 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-web-config" (OuterVolumeSpecName: "web-config") pod "a01f0b27-4507-412a-a5b3-76adda68a09d" (UID: "a01f0b27-4507-412a-a5b3-76adda68a09d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 15:38:55.766647 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766586 2557 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.766647 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766610 2557 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.766647 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766619 2557 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-kube-rbac-proxy\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.766647 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766629 2557 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a01f0b27-4507-412a-a5b3-76adda68a09d-tls-assets\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.766647 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766637 2557 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-config\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.766647 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766647 2557 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-web-config\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.767099 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766655 2557 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.767099 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766675 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-db\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.767099 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766685 2557 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-metrics-client-ca\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.767099 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766696 2557 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a01f0b27-4507-412a-a5b3-76adda68a09d-config-out\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.767099 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766704 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.767099 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766712 2557 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-grpc-tls\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.767099 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766720 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.767099 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766727 2557 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.767099 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766736 2557 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.767099 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766744 2557 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-metrics-client-certs\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.767099 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766752 2557 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a01f0b27-4507-412a-a5b3-76adda68a09d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.767099 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766760 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn68s\" (UniqueName: \"kubernetes.io/projected/a01f0b27-4507-412a-a5b3-76adda68a09d-kube-api-access-cn68s\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.767099 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766768 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:55.767099 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:55.766776 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a01f0b27-4507-412a-a5b3-76adda68a09d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:38:56.475345 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.475305 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a01f0b27-4507-412a-a5b3-76adda68a09d","Type":"ContainerDied","Data":"378c2fd8c1e90321fcc3b837ab375ffb8955b50ddce6091da6fb43ec8d0ee287"} Jun 08 15:38:56.475553 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.475369 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.475553 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.475398 2557 scope.go:117] "RemoveContainer" containerID="d758da6bd3932d97919c3455742c1aab6f342abbc3312c802785109433eb1ca2" Jun 08 15:38:56.483220 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.483186 2557 scope.go:117] "RemoveContainer" containerID="e9cde7715a8d3f606631753c14e2620c2ce732da76f80b73a6a8a8391db62f6f" Jun 08 15:38:56.490076 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.490061 2557 scope.go:117] "RemoveContainer" containerID="eecfedbdf30efb8006520c4f211bcad8510d713d399c69ded931e3a896d1bb9f" Jun 08 15:38:56.495702 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.495680 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 15:38:56.497584 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.497572 2557 scope.go:117] "RemoveContainer" containerID="d1c30ec61f54ef8deb4d031e610aba759a66dc259c95ba94b85ffa7a1cc3892d" Jun 08 15:38:56.504165 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.504114 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 15:38:56.505164 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.505150 2557 scope.go:117] "RemoveContainer" containerID="bae926091380ea4def07a2218dca672ba10c8c039f41230a8d89ef16743d1fae" Jun 08 15:38:56.511751 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.511735 2557 scope.go:117] "RemoveContainer" containerID="b842fa9e62d6f3eca2fb6f6ee178ceb9fc1e2c897f3c94e9682d67468b35149b" Jun 08 15:38:56.518530 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.518513 2557 scope.go:117] "RemoveContainer" containerID="5b5a817f60c7e3dfa33f4917675484ac6ac6d51eb098529d5cc246bae934cbd1" Jun 08 15:38:56.528441 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528421 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 15:38:56.528718 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528705 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="init-config-reloader" Jun 08 15:38:56.528763 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528720 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="init-config-reloader" Jun 08 15:38:56.528763 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528729 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="kube-rbac-proxy" Jun 08 15:38:56.528763 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528735 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="kube-rbac-proxy" Jun 08 15:38:56.528763 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528743 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="kube-rbac-proxy-web" Jun 08 15:38:56.528763 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528748 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="kube-rbac-proxy-web" Jun 08 15:38:56.528763 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528762 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="config-reloader" Jun 08 15:38:56.528956 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528769 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="config-reloader" Jun 08 15:38:56.528956 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528774 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="thanos-sidecar" Jun 08 15:38:56.528956 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528779 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="thanos-sidecar" Jun 08 15:38:56.528956 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528784 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="kube-rbac-proxy-thanos" Jun 08 15:38:56.528956 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528789 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="kube-rbac-proxy-thanos" Jun 08 15:38:56.528956 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528802 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="prometheus" Jun 08 15:38:56.528956 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528806 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="prometheus" Jun 08 15:38:56.528956 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528850 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="thanos-sidecar" Jun 08 15:38:56.528956 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528859 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="config-reloader" Jun 08 15:38:56.528956 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528865 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="kube-rbac-proxy-thanos" Jun 08 15:38:56.528956 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528871 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="prometheus" Jun 08 15:38:56.528956 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528877 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="kube-rbac-proxy-web" Jun 08 15:38:56.528956 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.528883 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" containerName="kube-rbac-proxy" Jun 08 15:38:56.534690 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.534672 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.537132 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.537109 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 08 15:38:56.537245 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.537113 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 08 15:38:56.537245 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.537147 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 08 15:38:56.537245 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.537113 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 08 15:38:56.537245 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.537111 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 08 15:38:56.537538 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.537523 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 08 15:38:56.537761 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.537742 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 08 15:38:56.537761 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.537778 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v9qwf\"" Jun 08 15:38:56.537957 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.537940 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 08 15:38:56.538281 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.538246 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 08 15:38:56.538386 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.538347 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 08 15:38:56.538386 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.538369 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 08 15:38:56.538534 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.538392 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 08 15:38:56.538660 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.538643 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 08 15:38:56.538729 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.538687 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d3nt4ki9tvnek\"" Jun 08 15:38:56.539854 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.539836 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 08 15:38:56.543583 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.543566 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 08 15:38:56.552051 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.552030 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 15:38:56.573028 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.572999 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82738f33-4db5-49b0-8d50-7012c6b4e13f-config-out\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573131 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573031 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573131 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573060 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573131 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573124 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-config\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573251 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573147 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573251 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573204 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573251 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573227 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573446 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573261 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573446 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573311 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573446 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573345 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz8bg\" (UniqueName: \"kubernetes.io/projected/82738f33-4db5-49b0-8d50-7012c6b4e13f-kube-api-access-xz8bg\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573446 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573369 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573446 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573407 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82738f33-4db5-49b0-8d50-7012c6b4e13f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573446 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573427 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573689 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573488 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82738f33-4db5-49b0-8d50-7012c6b4e13f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573689 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573508 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573689 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573565 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-web-config\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573689 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573600 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573689 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573620 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573689 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573664 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.573863 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.573695 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674014 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.673990 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82738f33-4db5-49b0-8d50-7012c6b4e13f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674138 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674018 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674138 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674042 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82738f33-4db5-49b0-8d50-7012c6b4e13f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674138 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674062 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674138 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674083 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-web-config\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674138 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674103 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674138 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674118 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674138 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674135 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674564 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674151 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674564 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674166 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82738f33-4db5-49b0-8d50-7012c6b4e13f-config-out\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674564 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674204 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674564 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674233 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674564 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674273 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-config\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674564 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674297 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674564 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674332 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674564 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674355 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674564 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674397 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674564 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674439 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674564 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674501 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz8bg\" (UniqueName: \"kubernetes.io/projected/82738f33-4db5-49b0-8d50-7012c6b4e13f-kube-api-access-xz8bg\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.674564 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674530 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.675134 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674910 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.675134 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.674920 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.675134 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.675052 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.675287 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.675201 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82738f33-4db5-49b0-8d50-7012c6b4e13f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.675956 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.675913 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.677490 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.677060 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.678338 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.677880 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.678338 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.677925 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82738f33-4db5-49b0-8d50-7012c6b4e13f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.678338 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.678303 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.678531 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.678376 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.678531 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.678453 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82738f33-4db5-49b0-8d50-7012c6b4e13f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.678897 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.678847 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.678897 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.678890 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.679336 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.679311 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-web-config\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.679444 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.679426 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.680002 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.679972 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.680002 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.679987 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82738f33-4db5-49b0-8d50-7012c6b4e13f-config-out\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.680579 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.680559 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.680952 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.680937 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/82738f33-4db5-49b0-8d50-7012c6b4e13f-config\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.682405 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.682385 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz8bg\" (UniqueName: \"kubernetes.io/projected/82738f33-4db5-49b0-8d50-7012c6b4e13f-kube-api-access-xz8bg\") pod \"prometheus-k8s-0\" (UID: \"82738f33-4db5-49b0-8d50-7012c6b4e13f\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.844602 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.844526 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:38:56.990261 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:56.990227 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 15:38:56.993695 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:38:56.993662 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82738f33_4db5_49b0_8d50_7012c6b4e13f.slice/crio-a908256f357929b4b5e0b9094eeb89458f68069b9251f4d7f745f0c61f5bb729 WatchSource:0}: Error finding container a908256f357929b4b5e0b9094eeb89458f68069b9251f4d7f745f0c61f5bb729: Status 404 returned error can't find the container with id a908256f357929b4b5e0b9094eeb89458f68069b9251f4d7f745f0c61f5bb729 Jun 08 15:38:57.481693 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:57.481659 2557 generic.go:356] "Generic (PLEG): container finished" podID="82738f33-4db5-49b0-8d50-7012c6b4e13f" containerID="8634c00dcfb5fbe0575e126e0dd2bb23be5e52faedb3459ee4e92aed574f4e41" exitCode=0 Jun 08 15:38:57.481876 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:57.481742 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82738f33-4db5-49b0-8d50-7012c6b4e13f","Type":"ContainerDied","Data":"8634c00dcfb5fbe0575e126e0dd2bb23be5e52faedb3459ee4e92aed574f4e41"} Jun 08 15:38:57.481876 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:57.481816 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82738f33-4db5-49b0-8d50-7012c6b4e13f","Type":"ContainerStarted","Data":"a908256f357929b4b5e0b9094eeb89458f68069b9251f4d7f745f0c61f5bb729"} Jun 08 15:38:58.338272 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:58.338231 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a01f0b27-4507-412a-a5b3-76adda68a09d" path="/var/lib/kubelet/pods/a01f0b27-4507-412a-a5b3-76adda68a09d/volumes" Jun 08 15:38:58.489587 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:58.489556 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82738f33-4db5-49b0-8d50-7012c6b4e13f","Type":"ContainerStarted","Data":"104cf418d19e57ff5ff0ac761f5365175d8aeed854dab78d789b61f08d0ff6d0"} Jun 08 15:38:58.489743 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:58.489620 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82738f33-4db5-49b0-8d50-7012c6b4e13f","Type":"ContainerStarted","Data":"071aec9298706587c977a12d489fd4e6897b1a87683c819b8806815a0b5705b4"} Jun 08 15:38:58.489743 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:58.489645 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82738f33-4db5-49b0-8d50-7012c6b4e13f","Type":"ContainerStarted","Data":"e9616a27e1d6e195bc9d9312255884d34f3dbb9948e2da284c57cc02eb126c48"} Jun 08 15:38:58.489743 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:58.489668 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82738f33-4db5-49b0-8d50-7012c6b4e13f","Type":"ContainerStarted","Data":"df7cee7745c3edca452dab19f1fe895e148e53d4b2481bc89b5a279b2fa7adc4"} Jun 08 15:38:58.489743 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:58.489692 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82738f33-4db5-49b0-8d50-7012c6b4e13f","Type":"ContainerStarted","Data":"c424dbafdfa6375c7ff303f006a70a7fcadbccf45cafe75b90f37b544f231a64"} Jun 08 15:38:58.489743 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:58.489714 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82738f33-4db5-49b0-8d50-7012c6b4e13f","Type":"ContainerStarted","Data":"a0e360500deff27dfc5a8d669d9beba8ebe9026fb6c7bb6da12b01ed6ebd4c92"} Jun 08 15:38:58.537694 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:58.537635 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.517088259 podStartE2EDuration="2.537615026s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.526767ms" isStatefulPod=false podCreationTimestamp="2026-06-08 15:38:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:38:58.535399882 +0000 UTC m=+328.624530934" watchObservedRunningTime="2026-06-08 15:38:58.537615026 +0000 UTC m=+328.626746077" Jun 08 15:38:58.539817 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:58.539789 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:58.599333 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:58.599262 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:58.658921 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:58.658889 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:58.762312 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:58.762283 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:58.865380 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:58.865306 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:59.246728 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:59.246677 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:38:59.308493 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:38:59.308444 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:39:01.844849 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:01.844819 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:39:02.296550 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:02.296492 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:39:02.347675 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:02.347638 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:39:03.725841 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:03.725812 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:39:03.826402 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:03.826362 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:39:08.659162 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.659118 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" podUID="a0a2ada9-1340-40f3-b339-f2a682aecf8d" containerName="registry" containerID="cri-o://0d15f9a4c709e94b86806309a82a02443fbdde835e850a971404bc3d0f8a9fa9" gracePeriod=30 Jun 08 15:39:08.901634 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.901613 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:39:08.975864 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.975770 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls\") pod \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " Jun 08 15:39:08.975864 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.975818 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-bound-sa-token\") pod \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " Jun 08 15:39:08.975864 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.975859 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a0a2ada9-1340-40f3-b339-f2a682aecf8d-image-registry-private-configuration\") pod \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " Jun 08 15:39:08.976130 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.975904 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a0a2ada9-1340-40f3-b339-f2a682aecf8d-ca-trust-extracted\") pod \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " Jun 08 15:39:08.976130 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.975927 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7mzj\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-kube-api-access-r7mzj\") pod \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " Jun 08 15:39:08.976130 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.975996 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a0a2ada9-1340-40f3-b339-f2a682aecf8d-installation-pull-secrets\") pod \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " Jun 08 15:39:08.976130 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.976023 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-certificates\") pod \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " Jun 08 15:39:08.976130 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.976044 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0a2ada9-1340-40f3-b339-f2a682aecf8d-trusted-ca\") pod \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\" (UID: \"a0a2ada9-1340-40f3-b339-f2a682aecf8d\") " Jun 08 15:39:08.976639 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.976614 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0a2ada9-1340-40f3-b339-f2a682aecf8d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a0a2ada9-1340-40f3-b339-f2a682aecf8d" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 15:39:08.976740 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.976665 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a0a2ada9-1340-40f3-b339-f2a682aecf8d" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 15:39:08.978573 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.978519 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a0a2ada9-1340-40f3-b339-f2a682aecf8d" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 15:39:08.978721 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.978652 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0a2ada9-1340-40f3-b339-f2a682aecf8d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a0a2ada9-1340-40f3-b339-f2a682aecf8d" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 15:39:08.978721 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.978658 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a0a2ada9-1340-40f3-b339-f2a682aecf8d" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 15:39:08.978869 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.978844 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0a2ada9-1340-40f3-b339-f2a682aecf8d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a0a2ada9-1340-40f3-b339-f2a682aecf8d" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 15:39:08.979115 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.979092 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-kube-api-access-r7mzj" (OuterVolumeSpecName: "kube-api-access-r7mzj") pod "a0a2ada9-1340-40f3-b339-f2a682aecf8d" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d"). InnerVolumeSpecName "kube-api-access-r7mzj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 15:39:08.984710 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:08.984685 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0a2ada9-1340-40f3-b339-f2a682aecf8d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a0a2ada9-1340-40f3-b339-f2a682aecf8d" (UID: "a0a2ada9-1340-40f3-b339-f2a682aecf8d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 15:39:09.076572 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.076535 2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a0a2ada9-1340-40f3-b339-f2a682aecf8d-ca-trust-extracted\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:39:09.076572 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.076564 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7mzj\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-kube-api-access-r7mzj\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:39:09.076572 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.076575 2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a0a2ada9-1340-40f3-b339-f2a682aecf8d-installation-pull-secrets\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:39:09.076783 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.076584 2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-certificates\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:39:09.076783 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.076594 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0a2ada9-1340-40f3-b339-f2a682aecf8d-trusted-ca\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:39:09.076783 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.076602 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-registry-tls\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:39:09.076783 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.076610 2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0a2ada9-1340-40f3-b339-f2a682aecf8d-bound-sa-token\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:39:09.076783 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.076619 2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a0a2ada9-1340-40f3-b339-f2a682aecf8d-image-registry-private-configuration\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:39:09.534781 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.534750 2557 generic.go:356] "Generic (PLEG): container finished" podID="a0a2ada9-1340-40f3-b339-f2a682aecf8d" containerID="0d15f9a4c709e94b86806309a82a02443fbdde835e850a971404bc3d0f8a9fa9" exitCode=0 Jun 08 15:39:09.534964 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.534789 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" Jun 08 15:39:09.534964 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.534809 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" event={"ID":"a0a2ada9-1340-40f3-b339-f2a682aecf8d","Type":"ContainerDied","Data":"0d15f9a4c709e94b86806309a82a02443fbdde835e850a971404bc3d0f8a9fa9"} Jun 08 15:39:09.534964 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.534847 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f8f6f6b6-jx5jx" event={"ID":"a0a2ada9-1340-40f3-b339-f2a682aecf8d","Type":"ContainerDied","Data":"5cf93dfac92b04aa9de12bd148ea5372d2c97e1daeb1c2aee09532c64f6ffcbb"} Jun 08 15:39:09.534964 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.534866 2557 scope.go:117] "RemoveContainer" containerID="0d15f9a4c709e94b86806309a82a02443fbdde835e850a971404bc3d0f8a9fa9" Jun 08 15:39:09.543872 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.543852 2557 scope.go:117] "RemoveContainer" containerID="0d15f9a4c709e94b86806309a82a02443fbdde835e850a971404bc3d0f8a9fa9" Jun 08 15:39:09.544149 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:39:09.544129 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d15f9a4c709e94b86806309a82a02443fbdde835e850a971404bc3d0f8a9fa9\": container with ID starting with 0d15f9a4c709e94b86806309a82a02443fbdde835e850a971404bc3d0f8a9fa9 not found: ID does not exist" containerID="0d15f9a4c709e94b86806309a82a02443fbdde835e850a971404bc3d0f8a9fa9" Jun 08 15:39:09.544216 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.544157 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d15f9a4c709e94b86806309a82a02443fbdde835e850a971404bc3d0f8a9fa9"} err="failed to get container status \"0d15f9a4c709e94b86806309a82a02443fbdde835e850a971404bc3d0f8a9fa9\": rpc error: code = NotFound desc = could not find container \"0d15f9a4c709e94b86806309a82a02443fbdde835e850a971404bc3d0f8a9fa9\": container with ID starting with 0d15f9a4c709e94b86806309a82a02443fbdde835e850a971404bc3d0f8a9fa9 not found: ID does not exist" Jun 08 15:39:09.554798 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.554777 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84f8f6f6b6-jx5jx"] Jun 08 15:39:09.557416 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.557393 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-84f8f6f6b6-jx5jx"] Jun 08 15:39:09.863310 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.863230 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:39:09.960934 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:09.960898 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:39:10.012280 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:10.012247 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:39:10.337665 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:10.337636 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0a2ada9-1340-40f3-b339-f2a682aecf8d" path="/var/lib/kubelet/pods/a0a2ada9-1340-40f3-b339-f2a682aecf8d/volumes" Jun 08 15:39:17.294723 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:17.294630 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:39:17.347659 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:17.347625 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:39:33.049286 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:33.049251 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:39:33.150942 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:33.150907 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:39:56.845258 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:56.845230 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:39:56.860409 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:56.860385 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:39:57.732761 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:39:57.732734 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 15:40:05.090548 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:40:05.090511 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:40:05.140942 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:40:05.140911 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:40:16.919526 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:40:16.919487 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:40:17.020813 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:40:17.020784 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:40:32.299438 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:40:32.299395 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:40:32.397655 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:40:32.397620 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:40:32.720893 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:40:32.720859 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:40:44.076040 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:40:44.075951 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:40:55.260843 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:40:55.260810 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:40:55.313870 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:40:55.313835 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:40:55.368113 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:40:55.368076 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:42:07.847705 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:42:07.847673 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:42:28.691733 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:42:28.691652 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:42:46.828166 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:42:46.828117 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:18.272839 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:18.272806 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:18.371572 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:18.371534 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:30.307108 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:30.307080 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 15:43:30.310828 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:30.310806 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 15:43:30.315005 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:30.314976 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 15:43:30.318742 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:30.318725 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 15:43:39.557665 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:39.557629 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:40.928686 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:40.928652 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:43.163350 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:43.163309 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:43.263830 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:43.263800 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:45.887194 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:45.887155 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:45.970639 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:45.970604 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:46.127297 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:46.127267 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:46.237066 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:46.236976 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:49.360054 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:49.360024 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:49.730449 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:49.730415 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:50.359226 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:50.359191 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:50.409235 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:50.409196 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:55.735071 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:55.735038 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:55.821818 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:55.821775 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:43:56.950385 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:43:56.950354 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:14.372512 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:14.372455 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:14.470616 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:14.470580 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:16.336592 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:16.336559 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:16.444319 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:16.444287 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:16.496167 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:16.496134 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:21.561198 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:21.561164 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:21.662015 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:21.661988 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:23.441980 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:23.441938 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:27.196006 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.195958 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:27.243346 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.243309 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:27.334250 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.334218 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:27.405574 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.405549 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:27.701979 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.701952 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:27.747293 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.747265 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-876584476-p5p64"] Jun 08 15:44:27.747616 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.747601 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0a2ada9-1340-40f3-b339-f2a682aecf8d" containerName="registry" Jun 08 15:44:27.747669 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.747617 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0a2ada9-1340-40f3-b339-f2a682aecf8d" containerName="registry" Jun 08 15:44:27.747703 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.747687 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0a2ada9-1340-40f3-b339-f2a682aecf8d" containerName="registry" Jun 08 15:44:27.750365 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.750350 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" Jun 08 15:44:27.753542 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.753516 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 08 15:44:27.753672 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.753645 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 08 15:44:27.753745 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.753647 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 08 15:44:27.753805 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.753758 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 08 15:44:27.753805 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.753766 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bh9nv\"" Jun 08 15:44:27.753805 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.753772 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 08 15:44:27.759135 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.759111 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-876584476-p5p64"] Jun 08 15:44:27.782347 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.782320 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:27.824363 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.824332 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6309c02c-b717-4e1e-b61d-66f15d87e0ca-cert\") pod \"lws-controller-manager-876584476-p5p64\" (UID: \"6309c02c-b717-4e1e-b61d-66f15d87e0ca\") " pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" Jun 08 15:44:27.824529 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.824375 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6309c02c-b717-4e1e-b61d-66f15d87e0ca-manager-config\") pod \"lws-controller-manager-876584476-p5p64\" (UID: \"6309c02c-b717-4e1e-b61d-66f15d87e0ca\") " pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" Jun 08 15:44:27.824529 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.824403 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfbjn\" (UniqueName: \"kubernetes.io/projected/6309c02c-b717-4e1e-b61d-66f15d87e0ca-kube-api-access-cfbjn\") pod \"lws-controller-manager-876584476-p5p64\" (UID: \"6309c02c-b717-4e1e-b61d-66f15d87e0ca\") " pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" Jun 08 15:44:27.824529 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.824488 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6309c02c-b717-4e1e-b61d-66f15d87e0ca-metrics-cert\") pod \"lws-controller-manager-876584476-p5p64\" (UID: \"6309c02c-b717-4e1e-b61d-66f15d87e0ca\") " pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" Jun 08 15:44:27.925327 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.925296 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6309c02c-b717-4e1e-b61d-66f15d87e0ca-cert\") pod \"lws-controller-manager-876584476-p5p64\" (UID: \"6309c02c-b717-4e1e-b61d-66f15d87e0ca\") " pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" Jun 08 15:44:27.925445 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.925335 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6309c02c-b717-4e1e-b61d-66f15d87e0ca-manager-config\") pod \"lws-controller-manager-876584476-p5p64\" (UID: \"6309c02c-b717-4e1e-b61d-66f15d87e0ca\") " pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" Jun 08 15:44:27.925445 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.925359 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfbjn\" (UniqueName: \"kubernetes.io/projected/6309c02c-b717-4e1e-b61d-66f15d87e0ca-kube-api-access-cfbjn\") pod \"lws-controller-manager-876584476-p5p64\" (UID: \"6309c02c-b717-4e1e-b61d-66f15d87e0ca\") " pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" Jun 08 15:44:27.925445 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.925378 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6309c02c-b717-4e1e-b61d-66f15d87e0ca-metrics-cert\") pod \"lws-controller-manager-876584476-p5p64\" (UID: \"6309c02c-b717-4e1e-b61d-66f15d87e0ca\") " pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" Jun 08 15:44:27.925961 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.925941 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6309c02c-b717-4e1e-b61d-66f15d87e0ca-manager-config\") pod \"lws-controller-manager-876584476-p5p64\" (UID: \"6309c02c-b717-4e1e-b61d-66f15d87e0ca\") " pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" Jun 08 15:44:27.927921 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.927897 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6309c02c-b717-4e1e-b61d-66f15d87e0ca-cert\") pod \"lws-controller-manager-876584476-p5p64\" (UID: \"6309c02c-b717-4e1e-b61d-66f15d87e0ca\") " pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" Jun 08 15:44:27.927921 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.927914 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6309c02c-b717-4e1e-b61d-66f15d87e0ca-metrics-cert\") pod \"lws-controller-manager-876584476-p5p64\" (UID: \"6309c02c-b717-4e1e-b61d-66f15d87e0ca\") " pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" Jun 08 15:44:27.933348 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.933322 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:27.933781 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.933759 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfbjn\" (UniqueName: \"kubernetes.io/projected/6309c02c-b717-4e1e-b61d-66f15d87e0ca-kube-api-access-cfbjn\") pod \"lws-controller-manager-876584476-p5p64\" (UID: \"6309c02c-b717-4e1e-b61d-66f15d87e0ca\") " pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" Jun 08 15:44:27.992828 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:27.992768 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:28.060614 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:28.060592 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" Jun 08 15:44:28.187319 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:28.187134 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-876584476-p5p64"] Jun 08 15:44:28.192055 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:44:28.192027 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6309c02c_b717_4e1e_b61d_66f15d87e0ca.slice/crio-68687683daf4ca3015ba32192bb1954a768c8456f241f028942047235230d977 WatchSource:0}: Error finding container 68687683daf4ca3015ba32192bb1954a768c8456f241f028942047235230d977: Status 404 returned error can't find the container with id 68687683daf4ca3015ba32192bb1954a768c8456f241f028942047235230d977 Jun 08 15:44:28.194213 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:28.194195 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 15:44:28.731385 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:28.731351 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" event={"ID":"6309c02c-b717-4e1e-b61d-66f15d87e0ca","Type":"ContainerStarted","Data":"68687683daf4ca3015ba32192bb1954a768c8456f241f028942047235230d977"} Jun 08 15:44:31.746766 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:31.746726 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" event={"ID":"6309c02c-b717-4e1e-b61d-66f15d87e0ca","Type":"ContainerStarted","Data":"808d105e603ed86b20e5d63a0bcd719d3bf4a01a83c9f92ec28a4b63b7d3c3c7"} Jun 08 15:44:31.747204 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:31.746869 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" Jun 08 15:44:31.765291 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:31.765236 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" podStartSLOduration=2.100733555 podStartE2EDuration="4.765222476s" totalImagesPullingTime="2.664488921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:44:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:44:31.764229902 +0000 UTC m=+661.853360955" watchObservedRunningTime="2026-06-08 15:44:31.765222476 +0000 UTC m=+661.854353526" Jun 08 15:44:35.368589 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:35.368554 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:35.475600 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:35.475562 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:35.980755 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:35.980720 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:36.183422 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.183384 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:36.282092 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.281996 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:36.309800 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.309764 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d"] Jun 08 15:44:36.313323 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.313305 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.316236 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.316215 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 08 15:44:36.316379 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.316259 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 08 15:44:36.318414 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.318395 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 08 15:44:36.318653 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.318638 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dskpq\"" Jun 08 15:44:36.318722 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.318668 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 08 15:44:36.338145 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.338121 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d"] Jun 08 15:44:36.400274 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.400243 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cffbdb15-0b61-4124-8f64-2a22b45e65a2-manifests\") pod \"opendatahub-operator-controller-manager-6484fb6dbf-qfw6d\" (UID: \"cffbdb15-0b61-4124-8f64-2a22b45e65a2\") " pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.400698 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.400310 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cffbdb15-0b61-4124-8f64-2a22b45e65a2-webhook-cert\") pod \"opendatahub-operator-controller-manager-6484fb6dbf-qfw6d\" (UID: \"cffbdb15-0b61-4124-8f64-2a22b45e65a2\") " pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.400698 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.400353 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj94v\" (UniqueName: \"kubernetes.io/projected/cffbdb15-0b61-4124-8f64-2a22b45e65a2-kube-api-access-zj94v\") pod \"opendatahub-operator-controller-manager-6484fb6dbf-qfw6d\" (UID: \"cffbdb15-0b61-4124-8f64-2a22b45e65a2\") " pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.400698 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.400382 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cffbdb15-0b61-4124-8f64-2a22b45e65a2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6484fb6dbf-qfw6d\" (UID: \"cffbdb15-0b61-4124-8f64-2a22b45e65a2\") " pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.400698 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.400427 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cffbdb15-0b61-4124-8f64-2a22b45e65a2-tmp\") pod \"opendatahub-operator-controller-manager-6484fb6dbf-qfw6d\" (UID: \"cffbdb15-0b61-4124-8f64-2a22b45e65a2\") " pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.427475 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.427441 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:36.501728 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.501693 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cffbdb15-0b61-4124-8f64-2a22b45e65a2-webhook-cert\") pod \"opendatahub-operator-controller-manager-6484fb6dbf-qfw6d\" (UID: \"cffbdb15-0b61-4124-8f64-2a22b45e65a2\") " pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.501879 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.501734 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj94v\" (UniqueName: \"kubernetes.io/projected/cffbdb15-0b61-4124-8f64-2a22b45e65a2-kube-api-access-zj94v\") pod \"opendatahub-operator-controller-manager-6484fb6dbf-qfw6d\" (UID: \"cffbdb15-0b61-4124-8f64-2a22b45e65a2\") " pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.501879 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.501756 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cffbdb15-0b61-4124-8f64-2a22b45e65a2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6484fb6dbf-qfw6d\" (UID: \"cffbdb15-0b61-4124-8f64-2a22b45e65a2\") " pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.501879 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.501792 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cffbdb15-0b61-4124-8f64-2a22b45e65a2-tmp\") pod \"opendatahub-operator-controller-manager-6484fb6dbf-qfw6d\" (UID: \"cffbdb15-0b61-4124-8f64-2a22b45e65a2\") " pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.501879 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.501816 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cffbdb15-0b61-4124-8f64-2a22b45e65a2-manifests\") pod \"opendatahub-operator-controller-manager-6484fb6dbf-qfw6d\" (UID: \"cffbdb15-0b61-4124-8f64-2a22b45e65a2\") " pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.502175 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.502144 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cffbdb15-0b61-4124-8f64-2a22b45e65a2-manifests\") pod \"opendatahub-operator-controller-manager-6484fb6dbf-qfw6d\" (UID: \"cffbdb15-0b61-4124-8f64-2a22b45e65a2\") " pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.504276 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.504247 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cffbdb15-0b61-4124-8f64-2a22b45e65a2-tmp\") pod \"opendatahub-operator-controller-manager-6484fb6dbf-qfw6d\" (UID: \"cffbdb15-0b61-4124-8f64-2a22b45e65a2\") " pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.504276 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.504260 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cffbdb15-0b61-4124-8f64-2a22b45e65a2-webhook-cert\") pod \"opendatahub-operator-controller-manager-6484fb6dbf-qfw6d\" (UID: \"cffbdb15-0b61-4124-8f64-2a22b45e65a2\") " pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.504406 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.504305 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cffbdb15-0b61-4124-8f64-2a22b45e65a2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6484fb6dbf-qfw6d\" (UID: \"cffbdb15-0b61-4124-8f64-2a22b45e65a2\") " pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.515338 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.515317 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj94v\" (UniqueName: \"kubernetes.io/projected/cffbdb15-0b61-4124-8f64-2a22b45e65a2-kube-api-access-zj94v\") pod \"opendatahub-operator-controller-manager-6484fb6dbf-qfw6d\" (UID: \"cffbdb15-0b61-4124-8f64-2a22b45e65a2\") " pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.537952 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.537891 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:36.623989 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.623960 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:36.633360 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.633338 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:36.733403 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.733367 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:36.761085 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.761053 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d"] Jun 08 15:44:36.766271 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:44:36.766248 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcffbdb15_0b61_4124_8f64_2a22b45e65a2.slice/crio-1665d6e844176d931042e2f83564df1da8cd88098e7fc31e88346cb39c95bc5a WatchSource:0}: Error finding container 1665d6e844176d931042e2f83564df1da8cd88098e7fc31e88346cb39c95bc5a: Status 404 returned error can't find the container with id 1665d6e844176d931042e2f83564df1da8cd88098e7fc31e88346cb39c95bc5a Jun 08 15:44:36.812271 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.812204 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:36.881016 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:36.880988 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:37.771890 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:37.771846 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" event={"ID":"cffbdb15-0b61-4124-8f64-2a22b45e65a2","Type":"ContainerStarted","Data":"1665d6e844176d931042e2f83564df1da8cd88098e7fc31e88346cb39c95bc5a"} Jun 08 15:44:38.090197 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:38.090107 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:38.190886 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:38.190805 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:38.291002 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:38.290965 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:39.783577 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:39.783548 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" event={"ID":"cffbdb15-0b61-4124-8f64-2a22b45e65a2","Type":"ContainerStarted","Data":"83a7c9b037bb7cc6b1c6434e7999cc71c858d2cf1c5232c8075535fddcb01c2f"} Jun 08 15:44:41.793498 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:41.793445 2557 generic.go:356] "Generic (PLEG): container finished" podID="cffbdb15-0b61-4124-8f64-2a22b45e65a2" containerID="83a7c9b037bb7cc6b1c6434e7999cc71c858d2cf1c5232c8075535fddcb01c2f" exitCode=0 Jun 08 15:44:41.793894 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:41.793516 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" event={"ID":"cffbdb15-0b61-4124-8f64-2a22b45e65a2","Type":"ContainerDied","Data":"83a7c9b037bb7cc6b1c6434e7999cc71c858d2cf1c5232c8075535fddcb01c2f"} Jun 08 15:44:42.752878 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:42.752851 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-876584476-p5p64" Jun 08 15:44:42.800254 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:42.800218 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" event={"ID":"cffbdb15-0b61-4124-8f64-2a22b45e65a2","Type":"ContainerStarted","Data":"ffaff94157a5350a457b960c621ac5a3661aae80cc33c6eb5c6ec85fc501bd07"} Jun 08 15:44:42.800657 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:42.800502 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:42.820042 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:42.819991 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" podStartSLOduration=2.488248898 podStartE2EDuration="6.819971394s" totalImagesPullingTime="3.266128556s" totalInitContainerRuntime="1.06559394s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:44:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:44:42.817782066 +0000 UTC m=+672.906913117" watchObservedRunningTime="2026-06-08 15:44:42.819971394 +0000 UTC m=+672.909102445" Jun 08 15:44:53.806788 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:53.806756 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6484fb6dbf-qfw6d" Jun 08 15:44:55.390967 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:55.390928 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:55.505047 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:55.505013 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:55.891217 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:55.891184 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:56.027011 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.026978 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:56.142495 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.142394 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-74547644b9-jwfsk"] Jun 08 15:44:56.150515 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.150492 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74547644b9-jwfsk" Jun 08 15:44:56.152923 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.152896 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 08 15:44:56.153033 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.152938 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 08 15:44:56.176490 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.176449 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74547644b9-jwfsk"] Jun 08 15:44:56.176868 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.176845 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwm8g\" (UniqueName: \"kubernetes.io/projected/07c7d369-f33d-4f07-a50f-9d10c42d5002-kube-api-access-bwm8g\") pod \"kube-auth-proxy-74547644b9-jwfsk\" (UID: \"07c7d369-f33d-4f07-a50f-9d10c42d5002\") " pod="openshift-ingress/kube-auth-proxy-74547644b9-jwfsk" Jun 08 15:44:56.176966 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.176946 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07c7d369-f33d-4f07-a50f-9d10c42d5002-tls-certs\") pod \"kube-auth-proxy-74547644b9-jwfsk\" (UID: \"07c7d369-f33d-4f07-a50f-9d10c42d5002\") " pod="openshift-ingress/kube-auth-proxy-74547644b9-jwfsk" Jun 08 15:44:56.177052 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.177035 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07c7d369-f33d-4f07-a50f-9d10c42d5002-tmp\") pod \"kube-auth-proxy-74547644b9-jwfsk\" (UID: \"07c7d369-f33d-4f07-a50f-9d10c42d5002\") " pod="openshift-ingress/kube-auth-proxy-74547644b9-jwfsk" Jun 08 15:44:56.193574 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.193546 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:56.278085 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.278058 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwm8g\" (UniqueName: \"kubernetes.io/projected/07c7d369-f33d-4f07-a50f-9d10c42d5002-kube-api-access-bwm8g\") pod \"kube-auth-proxy-74547644b9-jwfsk\" (UID: \"07c7d369-f33d-4f07-a50f-9d10c42d5002\") " pod="openshift-ingress/kube-auth-proxy-74547644b9-jwfsk" Jun 08 15:44:56.278233 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.278099 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07c7d369-f33d-4f07-a50f-9d10c42d5002-tls-certs\") pod \"kube-auth-proxy-74547644b9-jwfsk\" (UID: \"07c7d369-f33d-4f07-a50f-9d10c42d5002\") " pod="openshift-ingress/kube-auth-proxy-74547644b9-jwfsk" Jun 08 15:44:56.278233 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.278154 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07c7d369-f33d-4f07-a50f-9d10c42d5002-tmp\") pod \"kube-auth-proxy-74547644b9-jwfsk\" (UID: \"07c7d369-f33d-4f07-a50f-9d10c42d5002\") " pod="openshift-ingress/kube-auth-proxy-74547644b9-jwfsk" Jun 08 15:44:56.280451 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.280420 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07c7d369-f33d-4f07-a50f-9d10c42d5002-tmp\") pod \"kube-auth-proxy-74547644b9-jwfsk\" (UID: \"07c7d369-f33d-4f07-a50f-9d10c42d5002\") " pod="openshift-ingress/kube-auth-proxy-74547644b9-jwfsk" Jun 08 15:44:56.280564 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.280548 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07c7d369-f33d-4f07-a50f-9d10c42d5002-tls-certs\") pod \"kube-auth-proxy-74547644b9-jwfsk\" (UID: \"07c7d369-f33d-4f07-a50f-9d10c42d5002\") " pod="openshift-ingress/kube-auth-proxy-74547644b9-jwfsk" Jun 08 15:44:56.287367 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.287342 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwm8g\" (UniqueName: \"kubernetes.io/projected/07c7d369-f33d-4f07-a50f-9d10c42d5002-kube-api-access-bwm8g\") pod \"kube-auth-proxy-74547644b9-jwfsk\" (UID: \"07c7d369-f33d-4f07-a50f-9d10c42d5002\") " pod="openshift-ingress/kube-auth-proxy-74547644b9-jwfsk" Jun 08 15:44:56.373252 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.373222 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:56.460108 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.460019 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74547644b9-jwfsk" Jun 08 15:44:56.482644 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.482620 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:56.595107 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.595072 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74547644b9-jwfsk"] Jun 08 15:44:56.598713 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:44:56.598689 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07c7d369_f33d_4f07_a50f_9d10c42d5002.slice/crio-1bfbf015954ef8a852c27a64253c436f64f65caf8f242357512ceeabc4b474ca WatchSource:0}: Error finding container 1bfbf015954ef8a852c27a64253c436f64f65caf8f242357512ceeabc4b474ca: Status 404 returned error can't find the container with id 1bfbf015954ef8a852c27a64253c436f64f65caf8f242357512ceeabc4b474ca Jun 08 15:44:56.616284 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.616260 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:56.701141 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.701105 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:56.863787 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:56.863705 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74547644b9-jwfsk" event={"ID":"07c7d369-f33d-4f07-a50f-9d10c42d5002","Type":"ContainerStarted","Data":"1bfbf015954ef8a852c27a64253c436f64f65caf8f242357512ceeabc4b474ca"} Jun 08 15:44:57.075547 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:57.075512 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:57.343195 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:57.343158 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:44:59.881898 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:59.881871 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74547644b9-jwfsk" event={"ID":"07c7d369-f33d-4f07-a50f-9d10c42d5002","Type":"ContainerStarted","Data":"398f9b95609ffd8e7847b5faa9fb932d6e724680131e0921c7dd4cd97f3942cf"} Jun 08 15:44:59.897871 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:44:59.897821 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-74547644b9-jwfsk" podStartSLOduration=0.89929479 podStartE2EDuration="3.897806343s" totalImagesPullingTime="2.998511553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:44:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:44:59.896344601 +0000 UTC m=+689.985475650" watchObservedRunningTime="2026-06-08 15:44:59.897806343 +0000 UTC m=+689.986937394" Jun 08 15:45:06.884092 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:06.884055 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:06.986796 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:06.986762 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:07.258128 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:07.258038 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:07.716656 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:07.716623 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:08.180503 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:08.180453 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:08.301017 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:08.300977 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:08.399390 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:08.399356 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:16.460628 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:16.460542 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:16.564489 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:16.564431 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:17.684213 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:17.684184 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:17.785170 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:17.785138 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:18.591032 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:18.591003 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:18.691932 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:18.691898 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:20.592185 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:20.592155 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:26.550479 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:26.550436 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:30.965574 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:30.965532 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:31.024520 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:31.024486 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:42.413623 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:42.413585 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:47.956699 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:47.956666 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:48.052981 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:48.052947 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:48.481934 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:48.481901 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:53.293115 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:53.293066 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:53.393210 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:53.393178 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:53.582476 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:53.582386 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:53.791697 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:53.791653 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:53.891803 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:53.891774 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:56.671122 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:56.671087 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:56.891874 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:56.891821 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:56.991984 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:56.991891 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:45:57.050315 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:45:57.050274 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:25.164885 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:25.164841 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:29.167075 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:29.167045 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:34.135686 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:34.135649 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r"] Jun 08 15:46:34.137913 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:34.137897 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:46:34.140812 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:34.140792 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 08 15:46:34.140913 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:34.140792 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v2p9l\"" Jun 08 15:46:34.140913 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:34.140886 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 08 15:46:34.160008 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:34.159985 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r"] Jun 08 15:46:34.212299 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:34.212274 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/135cb359-a17c-4faa-95a3-fd3759210ff6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dsc4r\" (UID: \"135cb359-a17c-4faa-95a3-fd3759210ff6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:46:34.212429 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:34.212309 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvfrc\" (UniqueName: \"kubernetes.io/projected/135cb359-a17c-4faa-95a3-fd3759210ff6-kube-api-access-cvfrc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dsc4r\" (UID: \"135cb359-a17c-4faa-95a3-fd3759210ff6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:46:34.313342 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:34.313314 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/135cb359-a17c-4faa-95a3-fd3759210ff6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dsc4r\" (UID: \"135cb359-a17c-4faa-95a3-fd3759210ff6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:46:34.313478 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:34.313349 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvfrc\" (UniqueName: \"kubernetes.io/projected/135cb359-a17c-4faa-95a3-fd3759210ff6-kube-api-access-cvfrc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dsc4r\" (UID: \"135cb359-a17c-4faa-95a3-fd3759210ff6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:46:34.313670 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:34.313653 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/135cb359-a17c-4faa-95a3-fd3759210ff6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dsc4r\" (UID: \"135cb359-a17c-4faa-95a3-fd3759210ff6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:46:34.326308 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:34.326285 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvfrc\" (UniqueName: \"kubernetes.io/projected/135cb359-a17c-4faa-95a3-fd3759210ff6-kube-api-access-cvfrc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dsc4r\" (UID: \"135cb359-a17c-4faa-95a3-fd3759210ff6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:46:34.448084 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:34.448005 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:46:34.578223 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:34.578202 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r"] Jun 08 15:46:34.581109 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:46:34.581087 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod135cb359_a17c_4faa_95a3_fd3759210ff6.slice/crio-96b8fbf98943b9c5260415f9c8dc55fde72a51f8d8b8a40a0ff3fe7b1e8fabb7 WatchSource:0}: Error finding container 96b8fbf98943b9c5260415f9c8dc55fde72a51f8d8b8a40a0ff3fe7b1e8fabb7: Status 404 returned error can't find the container with id 96b8fbf98943b9c5260415f9c8dc55fde72a51f8d8b8a40a0ff3fe7b1e8fabb7 Jun 08 15:46:34.948758 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:34.948721 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:35.281843 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:35.281761 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" event={"ID":"135cb359-a17c-4faa-95a3-fd3759210ff6","Type":"ContainerStarted","Data":"96b8fbf98943b9c5260415f9c8dc55fde72a51f8d8b8a40a0ff3fe7b1e8fabb7"} Jun 08 15:46:35.664858 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:35.664808 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:35.817708 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:35.817678 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:37.805049 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:37.804988 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:38.494881 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:38.494847 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:40.384618 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:40.384587 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:40.830397 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:40.830319 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:42.825513 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:42.825383 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:43.318721 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:43.318682 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" event={"ID":"135cb359-a17c-4faa-95a3-fd3759210ff6","Type":"ContainerStarted","Data":"34c661fedf799a70ad1cbecc751df45af47dc5d272d433267803e3e3980fc2f9"} Jun 08 15:46:43.318930 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:43.318911 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:46:43.341936 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:43.341890 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" podStartSLOduration=1.425426828 podStartE2EDuration="9.341872354s" totalImagesPullingTime="7.916445526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:46:43.340510544 +0000 UTC m=+793.429641595" watchObservedRunningTime="2026-06-08 15:46:43.341872354 +0000 UTC m=+793.431003404" Jun 08 15:46:45.319360 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.319327 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l"] Jun 08 15:46:45.321625 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.321598 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l" Jun 08 15:46:45.323159 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.323131 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:45.323767 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.323740 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 08 15:46:45.324168 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.324113 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gwhc2\"" Jun 08 15:46:45.324359 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.324346 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 08 15:46:45.331038 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.331016 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l"] Jun 08 15:46:45.410736 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.410707 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3b7867cd-55e9-4b0b-b6d0-5309761631aa-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5xz5l\" (UID: \"3b7867cd-55e9-4b0b-b6d0-5309761631aa\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l" Jun 08 15:46:45.410884 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.410761 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b7867cd-55e9-4b0b-b6d0-5309761631aa-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5xz5l\" (UID: \"3b7867cd-55e9-4b0b-b6d0-5309761631aa\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l" Jun 08 15:46:45.410884 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.410804 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc9vn\" (UniqueName: \"kubernetes.io/projected/3b7867cd-55e9-4b0b-b6d0-5309761631aa-kube-api-access-gc9vn\") pod \"kuadrant-console-plugin-66b5774985-5xz5l\" (UID: \"3b7867cd-55e9-4b0b-b6d0-5309761631aa\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l" Jun 08 15:46:45.512185 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.512158 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b7867cd-55e9-4b0b-b6d0-5309761631aa-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5xz5l\" (UID: \"3b7867cd-55e9-4b0b-b6d0-5309761631aa\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l" Jun 08 15:46:45.512348 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.512190 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc9vn\" (UniqueName: \"kubernetes.io/projected/3b7867cd-55e9-4b0b-b6d0-5309761631aa-kube-api-access-gc9vn\") pod \"kuadrant-console-plugin-66b5774985-5xz5l\" (UID: \"3b7867cd-55e9-4b0b-b6d0-5309761631aa\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l" Jun 08 15:46:45.512348 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.512264 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3b7867cd-55e9-4b0b-b6d0-5309761631aa-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5xz5l\" (UID: \"3b7867cd-55e9-4b0b-b6d0-5309761631aa\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l" Jun 08 15:46:45.512994 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.512975 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3b7867cd-55e9-4b0b-b6d0-5309761631aa-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5xz5l\" (UID: \"3b7867cd-55e9-4b0b-b6d0-5309761631aa\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l" Jun 08 15:46:45.514608 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.514591 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b7867cd-55e9-4b0b-b6d0-5309761631aa-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5xz5l\" (UID: \"3b7867cd-55e9-4b0b-b6d0-5309761631aa\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l" Jun 08 15:46:45.521643 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.521618 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc9vn\" (UniqueName: \"kubernetes.io/projected/3b7867cd-55e9-4b0b-b6d0-5309761631aa-kube-api-access-gc9vn\") pod \"kuadrant-console-plugin-66b5774985-5xz5l\" (UID: \"3b7867cd-55e9-4b0b-b6d0-5309761631aa\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l" Jun 08 15:46:45.549652 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.549626 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:45.616376 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.616297 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:45.648645 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.648622 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l" Jun 08 15:46:45.772540 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:45.772510 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l"] Jun 08 15:46:45.775564 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:46:45.775535 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b7867cd_55e9_4b0b_b6d0_5309761631aa.slice/crio-8ce3814ca0dfc65299f92d87800de9d8d09a4a6597c9da93d2bcf8f2dfca50e1 WatchSource:0}: Error finding container 8ce3814ca0dfc65299f92d87800de9d8d09a4a6597c9da93d2bcf8f2dfca50e1: Status 404 returned error can't find the container with id 8ce3814ca0dfc65299f92d87800de9d8d09a4a6597c9da93d2bcf8f2dfca50e1 Jun 08 15:46:46.336860 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:46.336827 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l" event={"ID":"3b7867cd-55e9-4b0b-b6d0-5309761631aa","Type":"ContainerStarted","Data":"8ce3814ca0dfc65299f92d87800de9d8d09a4a6597c9da93d2bcf8f2dfca50e1"} Jun 08 15:46:49.561201 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:49.561161 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:49.619031 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:49.618997 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:54.326152 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:54.326123 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:46:56.418277 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:56.418241 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r"] Jun 08 15:46:56.418758 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:56.418502 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" podUID="135cb359-a17c-4faa-95a3-fd3759210ff6" containerName="manager" containerID="cri-o://34c661fedf799a70ad1cbecc751df45af47dc5d272d433267803e3e3980fc2f9" gracePeriod=2 Jun 08 15:46:56.424129 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:56.424100 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r"] Jun 08 15:46:56.446721 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:56.446685 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n"] Jun 08 15:46:56.447192 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:56.447148 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="135cb359-a17c-4faa-95a3-fd3759210ff6" containerName="manager" Jun 08 15:46:56.447192 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:56.447166 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="135cb359-a17c-4faa-95a3-fd3759210ff6" containerName="manager" Jun 08 15:46:56.447326 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:56.447276 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="135cb359-a17c-4faa-95a3-fd3759210ff6" containerName="manager" Jun 08 15:46:56.451785 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:56.451704 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" Jun 08 15:46:56.453833 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:46:56.453799 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dsc4r\" is forbidden: User \"system:node:ip-10-0-134-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-81.ec2.internal' and this object" podUID="135cb359-a17c-4faa-95a3-fd3759210ff6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:46:56.471178 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:56.471101 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n"] Jun 08 15:46:56.514070 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:56.514035 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ctz8n\" (UID: \"bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" Jun 08 15:46:56.514229 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:56.514131 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz2vm\" (UniqueName: \"kubernetes.io/projected/bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f-kube-api-access-tz2vm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ctz8n\" (UID: \"bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" Jun 08 15:46:56.614870 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:56.614824 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ctz8n\" (UID: \"bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" Jun 08 15:46:56.615061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:56.614946 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz2vm\" (UniqueName: \"kubernetes.io/projected/bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f-kube-api-access-tz2vm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ctz8n\" (UID: \"bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" Jun 08 15:46:56.615302 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:56.615275 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ctz8n\" (UID: \"bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" Jun 08 15:46:56.630192 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:56.630162 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz2vm\" (UniqueName: \"kubernetes.io/projected/bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f-kube-api-access-tz2vm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ctz8n\" (UID: \"bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" Jun 08 15:46:56.793297 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:56.793216 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" Jun 08 15:46:59.008908 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:59.008870 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:59.104068 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:59.104031 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:46:59.206143 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:46:59.206105 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:01.418911 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:01.418859 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:01.517963 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:01.517907 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:05.785984 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:05.785947 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:05.888431 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:05.888387 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:09.390684 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:09.390649 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:09.492410 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:09.492367 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:10.318169 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.318133 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n"] Jun 08 15:47:10.319269 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:47:10.319243 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc6dc8c5_13dc_4096_9336_ad79c7b7ae9f.slice/crio-8152097d0695893a9f5af2f7f9317939e5e25bd69a7909a9a512091c9a26a0e7 WatchSource:0}: Error finding container 8152097d0695893a9f5af2f7f9317939e5e25bd69a7909a9a512091c9a26a0e7: Status 404 returned error can't find the container with id 8152097d0695893a9f5af2f7f9317939e5e25bd69a7909a9a512091c9a26a0e7 Jun 08 15:47:10.356067 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.355984 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:47:10.359005 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:47:10.358969 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dsc4r\" is forbidden: User \"system:node:ip-10-0-134-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-81.ec2.internal' and this object" podUID="135cb359-a17c-4faa-95a3-fd3759210ff6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:47:10.446379 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.446312 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvfrc\" (UniqueName: \"kubernetes.io/projected/135cb359-a17c-4faa-95a3-fd3759210ff6-kube-api-access-cvfrc\") pod \"135cb359-a17c-4faa-95a3-fd3759210ff6\" (UID: \"135cb359-a17c-4faa-95a3-fd3759210ff6\") " Jun 08 15:47:10.446379 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.446352 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/135cb359-a17c-4faa-95a3-fd3759210ff6-extensions-socket-volume\") pod \"135cb359-a17c-4faa-95a3-fd3759210ff6\" (UID: \"135cb359-a17c-4faa-95a3-fd3759210ff6\") " Jun 08 15:47:10.446938 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.446915 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/135cb359-a17c-4faa-95a3-fd3759210ff6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "135cb359-a17c-4faa-95a3-fd3759210ff6" (UID: "135cb359-a17c-4faa-95a3-fd3759210ff6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 15:47:10.448937 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.448902 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/135cb359-a17c-4faa-95a3-fd3759210ff6-kube-api-access-cvfrc" (OuterVolumeSpecName: "kube-api-access-cvfrc") pod "135cb359-a17c-4faa-95a3-fd3759210ff6" (UID: "135cb359-a17c-4faa-95a3-fd3759210ff6"). InnerVolumeSpecName "kube-api-access-cvfrc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 15:47:10.453749 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.453723 2557 generic.go:356] "Generic (PLEG): container finished" podID="135cb359-a17c-4faa-95a3-fd3759210ff6" containerID="34c661fedf799a70ad1cbecc751df45af47dc5d272d433267803e3e3980fc2f9" exitCode=0 Jun 08 15:47:10.453845 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.453780 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:47:10.453845 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.453812 2557 scope.go:117] "RemoveContainer" containerID="34c661fedf799a70ad1cbecc751df45af47dc5d272d433267803e3e3980fc2f9" Jun 08 15:47:10.455743 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.455718 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" event={"ID":"bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f","Type":"ContainerStarted","Data":"6715f8fdf808e042e1168b6c7d84ab945ed320f6cfac0d9114a9a2d2859cde09"} Jun 08 15:47:10.455842 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.455774 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" event={"ID":"bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f","Type":"ContainerStarted","Data":"8152097d0695893a9f5af2f7f9317939e5e25bd69a7909a9a512091c9a26a0e7"} Jun 08 15:47:10.455949 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.455862 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" Jun 08 15:47:10.456048 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:47:10.456026 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dsc4r\" is forbidden: User \"system:node:ip-10-0-134-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-81.ec2.internal' and this object" podUID="135cb359-a17c-4faa-95a3-fd3759210ff6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:47:10.474424 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.474383 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" podStartSLOduration=14.474370131 podStartE2EDuration="14.474370131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:46:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:47:10.472704554 +0000 UTC m=+820.561835603" watchObservedRunningTime="2026-06-08 15:47:10.474370131 +0000 UTC m=+820.563501182" Jun 08 15:47:10.474713 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:47:10.474691 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dsc4r\" is forbidden: User \"system:node:ip-10-0-134-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-81.ec2.internal' and this object" podUID="135cb359-a17c-4faa-95a3-fd3759210ff6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:47:10.476303 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:47:10.476281 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dsc4r\" is forbidden: User \"system:node:ip-10-0-134-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-81.ec2.internal' and this object" podUID="135cb359-a17c-4faa-95a3-fd3759210ff6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:47:10.547350 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.547326 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvfrc\" (UniqueName: \"kubernetes.io/projected/135cb359-a17c-4faa-95a3-fd3759210ff6-kube-api-access-cvfrc\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:47:10.547485 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.547349 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/135cb359-a17c-4faa-95a3-fd3759210ff6-extensions-socket-volume\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:47:10.622403 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.622383 2557 scope.go:117] "RemoveContainer" containerID="34c661fedf799a70ad1cbecc751df45af47dc5d272d433267803e3e3980fc2f9" Jun 08 15:47:10.622755 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:47:10.622726 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34c661fedf799a70ad1cbecc751df45af47dc5d272d433267803e3e3980fc2f9\": container with ID starting with 34c661fedf799a70ad1cbecc751df45af47dc5d272d433267803e3e3980fc2f9 not found: ID does not exist" containerID="34c661fedf799a70ad1cbecc751df45af47dc5d272d433267803e3e3980fc2f9" Jun 08 15:47:10.622859 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:10.622761 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34c661fedf799a70ad1cbecc751df45af47dc5d272d433267803e3e3980fc2f9"} err="failed to get container status \"34c661fedf799a70ad1cbecc751df45af47dc5d272d433267803e3e3980fc2f9\": rpc error: code = NotFound desc = could not find container \"34c661fedf799a70ad1cbecc751df45af47dc5d272d433267803e3e3980fc2f9\": container with ID starting with 34c661fedf799a70ad1cbecc751df45af47dc5d272d433267803e3e3980fc2f9 not found: ID does not exist" Jun 08 15:47:11.461245 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:11.461167 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l" event={"ID":"3b7867cd-55e9-4b0b-b6d0-5309761631aa","Type":"ContainerStarted","Data":"e5cad794700df3a563010d4aae438e90353bcc0f5500eaa3470c9e0148863366"} Jun 08 15:47:11.463480 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:47:11.463432 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dsc4r\" is forbidden: User \"system:node:ip-10-0-134-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-81.ec2.internal' and this object" podUID="135cb359-a17c-4faa-95a3-fd3759210ff6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dsc4r" Jun 08 15:47:11.480134 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:11.480092 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xz5l" podStartSLOduration=1.6022485770000001 podStartE2EDuration="26.480076497s" totalImagesPullingTime="24.87782792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:47:11.478506727 +0000 UTC m=+821.567637776" watchObservedRunningTime="2026-06-08 15:47:11.480076497 +0000 UTC m=+821.569207547" Jun 08 15:47:12.337912 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:12.337872 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="135cb359-a17c-4faa-95a3-fd3759210ff6" path="/var/lib/kubelet/pods/135cb359-a17c-4faa-95a3-fd3759210ff6/volumes" Jun 08 15:47:12.733891 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:12.733848 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n"] Jun 08 15:47:12.734277 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:12.734046 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" podUID="bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f" containerName="manager" containerID="cri-o://6715f8fdf808e042e1168b6c7d84ab945ed320f6cfac0d9114a9a2d2859cde09" gracePeriod=10 Jun 08 15:47:13.028152 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:13.028123 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" Jun 08 15:47:13.170506 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:13.170456 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f-extensions-socket-volume\") pod \"bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f\" (UID: \"bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f\") " Jun 08 15:47:13.170506 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:13.170509 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tz2vm\" (UniqueName: \"kubernetes.io/projected/bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f-kube-api-access-tz2vm\") pod \"bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f\" (UID: \"bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f\") " Jun 08 15:47:13.170815 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:13.170792 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f" (UID: "bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 15:47:13.172543 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:13.172516 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f-kube-api-access-tz2vm" (OuterVolumeSpecName: "kube-api-access-tz2vm") pod "bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f" (UID: "bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f"). InnerVolumeSpecName "kube-api-access-tz2vm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 15:47:13.271278 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:13.271217 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f-extensions-socket-volume\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:47:13.271278 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:13.271238 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tz2vm\" (UniqueName: \"kubernetes.io/projected/bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f-kube-api-access-tz2vm\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:47:13.473027 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:13.472989 2557 generic.go:356] "Generic (PLEG): container finished" podID="bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f" containerID="6715f8fdf808e042e1168b6c7d84ab945ed320f6cfac0d9114a9a2d2859cde09" exitCode=0 Jun 08 15:47:13.473027 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:13.473030 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" Jun 08 15:47:13.473280 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:13.473077 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" event={"ID":"bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f","Type":"ContainerDied","Data":"6715f8fdf808e042e1168b6c7d84ab945ed320f6cfac0d9114a9a2d2859cde09"} Jun 08 15:47:13.473280 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:13.473136 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n" event={"ID":"bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f","Type":"ContainerDied","Data":"8152097d0695893a9f5af2f7f9317939e5e25bd69a7909a9a512091c9a26a0e7"} Jun 08 15:47:13.473280 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:13.473168 2557 scope.go:117] "RemoveContainer" containerID="6715f8fdf808e042e1168b6c7d84ab945ed320f6cfac0d9114a9a2d2859cde09" Jun 08 15:47:13.481778 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:13.481759 2557 scope.go:117] "RemoveContainer" containerID="6715f8fdf808e042e1168b6c7d84ab945ed320f6cfac0d9114a9a2d2859cde09" Jun 08 15:47:13.482024 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:47:13.481995 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6715f8fdf808e042e1168b6c7d84ab945ed320f6cfac0d9114a9a2d2859cde09\": container with ID starting with 6715f8fdf808e042e1168b6c7d84ab945ed320f6cfac0d9114a9a2d2859cde09 not found: ID does not exist" containerID="6715f8fdf808e042e1168b6c7d84ab945ed320f6cfac0d9114a9a2d2859cde09" Jun 08 15:47:13.482083 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:13.482035 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6715f8fdf808e042e1168b6c7d84ab945ed320f6cfac0d9114a9a2d2859cde09"} err="failed to get container status \"6715f8fdf808e042e1168b6c7d84ab945ed320f6cfac0d9114a9a2d2859cde09\": rpc error: code = NotFound desc = could not find container \"6715f8fdf808e042e1168b6c7d84ab945ed320f6cfac0d9114a9a2d2859cde09\": container with ID starting with 6715f8fdf808e042e1168b6c7d84ab945ed320f6cfac0d9114a9a2d2859cde09 not found: ID does not exist" Jun 08 15:47:13.494334 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:13.494312 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n"] Jun 08 15:47:13.497573 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:13.497553 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ctz8n"] Jun 08 15:47:14.337953 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:14.337915 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f" path="/var/lib/kubelet/pods/bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f/volumes" Jun 08 15:47:19.586155 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:19.586114 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:19.685635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:19.685598 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:22.041390 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:22.041360 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:22.140574 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:22.140542 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:28.103258 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:28.103216 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:28.152794 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:28.152766 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:29.048968 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:29.048934 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:29.148500 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:29.148453 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:32.568997 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:32.568961 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:40.824740 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:40.824697 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:40.831543 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:40.831513 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mgvfq"] Jun 08 15:47:40.831897 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:40.831883 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f" containerName="manager" Jun 08 15:47:40.831940 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:40.831900 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f" containerName="manager" Jun 08 15:47:40.831974 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:40.831966 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc6dc8c5-13dc-4096-9336-ad79c7b7ae9f" containerName="manager" Jun 08 15:47:40.896719 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:40.896689 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mgvfq"] Jun 08 15:47:40.896869 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:40.896846 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" Jun 08 15:47:40.898800 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:40.898775 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 08 15:47:40.929045 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:40.929019 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mgvfq"] Jun 08 15:47:40.939318 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:40.939291 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:40.997204 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:40.997177 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:41.003363 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:41.003343 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbcrq\" (UniqueName: \"kubernetes.io/projected/5af48550-0725-4a4f-adc0-2caabd10d73b-kube-api-access-wbcrq\") pod \"limitador-limitador-64c94fd64f-mgvfq\" (UID: \"5af48550-0725-4a4f-adc0-2caabd10d73b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" Jun 08 15:47:41.003480 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:41.003393 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5af48550-0725-4a4f-adc0-2caabd10d73b-config-file\") pod \"limitador-limitador-64c94fd64f-mgvfq\" (UID: \"5af48550-0725-4a4f-adc0-2caabd10d73b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" Jun 08 15:47:41.104609 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:41.104529 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbcrq\" (UniqueName: \"kubernetes.io/projected/5af48550-0725-4a4f-adc0-2caabd10d73b-kube-api-access-wbcrq\") pod \"limitador-limitador-64c94fd64f-mgvfq\" (UID: \"5af48550-0725-4a4f-adc0-2caabd10d73b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" Jun 08 15:47:41.104609 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:41.104600 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5af48550-0725-4a4f-adc0-2caabd10d73b-config-file\") pod \"limitador-limitador-64c94fd64f-mgvfq\" (UID: \"5af48550-0725-4a4f-adc0-2caabd10d73b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" Jun 08 15:47:41.105193 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:41.105175 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5af48550-0725-4a4f-adc0-2caabd10d73b-config-file\") pod \"limitador-limitador-64c94fd64f-mgvfq\" (UID: \"5af48550-0725-4a4f-adc0-2caabd10d73b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" Jun 08 15:47:41.112506 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:41.112476 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbcrq\" (UniqueName: \"kubernetes.io/projected/5af48550-0725-4a4f-adc0-2caabd10d73b-kube-api-access-wbcrq\") pod \"limitador-limitador-64c94fd64f-mgvfq\" (UID: \"5af48550-0725-4a4f-adc0-2caabd10d73b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" Jun 08 15:47:41.208919 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:41.208895 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" Jun 08 15:47:41.341718 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:41.341686 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mgvfq"] Jun 08 15:47:41.344831 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:47:41.344796 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5af48550_0725_4a4f_adc0_2caabd10d73b.slice/crio-8d7974a5c7085eb72621419ab0c452cda4cfeb1280b406d895b68cd64acb7243 WatchSource:0}: Error finding container 8d7974a5c7085eb72621419ab0c452cda4cfeb1280b406d895b68cd64acb7243: Status 404 returned error can't find the container with id 8d7974a5c7085eb72621419ab0c452cda4cfeb1280b406d895b68cd64acb7243 Jun 08 15:47:41.600352 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:41.600319 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" event={"ID":"5af48550-0725-4a4f-adc0-2caabd10d73b","Type":"ContainerStarted","Data":"8d7974a5c7085eb72621419ab0c452cda4cfeb1280b406d895b68cd64acb7243"} Jun 08 15:47:41.672623 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:41.672570 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:41.776736 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:41.776693 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:41.863666 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:41.863600 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-krpmp"] Jun 08 15:47:41.868246 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:41.868229 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-krpmp" Jun 08 15:47:41.870483 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:41.870453 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jrwzp\"" Jun 08 15:47:41.879505 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:41.879455 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-krpmp"] Jun 08 15:47:42.012903 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:42.012871 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x7q8\" (UniqueName: \"kubernetes.io/projected/84c45185-4920-4cf9-a754-cd54f53b7ae9-kube-api-access-2x7q8\") pod \"authorino-75d556bd77-krpmp\" (UID: \"84c45185-4920-4cf9-a754-cd54f53b7ae9\") " pod="kuadrant-system/authorino-75d556bd77-krpmp" Jun 08 15:47:42.114366 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:42.114286 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x7q8\" (UniqueName: \"kubernetes.io/projected/84c45185-4920-4cf9-a754-cd54f53b7ae9-kube-api-access-2x7q8\") pod \"authorino-75d556bd77-krpmp\" (UID: \"84c45185-4920-4cf9-a754-cd54f53b7ae9\") " pod="kuadrant-system/authorino-75d556bd77-krpmp" Jun 08 15:47:42.122277 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:42.122251 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x7q8\" (UniqueName: \"kubernetes.io/projected/84c45185-4920-4cf9-a754-cd54f53b7ae9-kube-api-access-2x7q8\") pod \"authorino-75d556bd77-krpmp\" (UID: \"84c45185-4920-4cf9-a754-cd54f53b7ae9\") " pod="kuadrant-system/authorino-75d556bd77-krpmp" Jun 08 15:47:42.177796 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:42.177763 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-krpmp" Jun 08 15:47:42.350651 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:42.350572 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-krpmp"] Jun 08 15:47:42.607442 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:42.607410 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-krpmp" event={"ID":"84c45185-4920-4cf9-a754-cd54f53b7ae9","Type":"ContainerStarted","Data":"d008d2ecd868885b79eb51aa1020cd20fdedf6820e5a436ceb52d2ea73c8b400"} Jun 08 15:47:43.900060 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:43.900025 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:43.957088 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:43.957040 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:46.629154 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:46.629121 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" event={"ID":"5af48550-0725-4a4f-adc0-2caabd10d73b","Type":"ContainerStarted","Data":"2623c3fa28cf494b9c456c768fac4af18320cb091e3fd9307baeb56478bace5a"} Jun 08 15:47:46.629660 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:46.629274 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" Jun 08 15:47:46.630617 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:46.630596 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-krpmp" event={"ID":"84c45185-4920-4cf9-a754-cd54f53b7ae9","Type":"ContainerStarted","Data":"42e600e0fb9553a55999f53e42ad7a68338a3d0b36673165e6a9a0161b5e22fd"} Jun 08 15:47:46.643583 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:46.643540 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" podStartSLOduration=2.055747123 podStartE2EDuration="6.64352681s" totalImagesPullingTime="4.587779687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:47:46.643341364 +0000 UTC m=+856.732472423" watchObservedRunningTime="2026-06-08 15:47:46.64352681 +0000 UTC m=+856.732657860" Jun 08 15:47:46.656364 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:46.656318 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-krpmp" podStartSLOduration=2.067296539 podStartE2EDuration="5.65630175s" totalImagesPullingTime="3.589005211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:47:46.655650812 +0000 UTC m=+856.744781861" watchObservedRunningTime="2026-06-08 15:47:46.65630175 +0000 UTC m=+856.745432799" Jun 08 15:47:55.952596 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:55.952560 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mgvfq"] Jun 08 15:47:55.953055 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:55.952826 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" podUID="5af48550-0725-4a4f-adc0-2caabd10d73b" containerName="limitador" containerID="cri-o://2623c3fa28cf494b9c456c768fac4af18320cb091e3fd9307baeb56478bace5a" gracePeriod=30 Jun 08 15:47:55.953495 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:55.953410 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" Jun 08 15:47:56.675197 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:56.675162 2557 generic.go:356] "Generic (PLEG): container finished" podID="5af48550-0725-4a4f-adc0-2caabd10d73b" containerID="2623c3fa28cf494b9c456c768fac4af18320cb091e3fd9307baeb56478bace5a" exitCode=0 Jun 08 15:47:56.675375 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:56.675235 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" event={"ID":"5af48550-0725-4a4f-adc0-2caabd10d73b","Type":"ContainerDied","Data":"2623c3fa28cf494b9c456c768fac4af18320cb091e3fd9307baeb56478bace5a"} Jun 08 15:47:56.892580 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:56.892554 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" Jun 08 15:47:57.038817 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:57.038733 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbcrq\" (UniqueName: \"kubernetes.io/projected/5af48550-0725-4a4f-adc0-2caabd10d73b-kube-api-access-wbcrq\") pod \"5af48550-0725-4a4f-adc0-2caabd10d73b\" (UID: \"5af48550-0725-4a4f-adc0-2caabd10d73b\") " Jun 08 15:47:57.038817 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:57.038802 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5af48550-0725-4a4f-adc0-2caabd10d73b-config-file\") pod \"5af48550-0725-4a4f-adc0-2caabd10d73b\" (UID: \"5af48550-0725-4a4f-adc0-2caabd10d73b\") " Jun 08 15:47:57.039279 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:57.039135 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5af48550-0725-4a4f-adc0-2caabd10d73b-config-file" (OuterVolumeSpecName: "config-file") pod "5af48550-0725-4a4f-adc0-2caabd10d73b" (UID: "5af48550-0725-4a4f-adc0-2caabd10d73b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 15:47:57.040826 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:57.040802 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5af48550-0725-4a4f-adc0-2caabd10d73b-kube-api-access-wbcrq" (OuterVolumeSpecName: "kube-api-access-wbcrq") pod "5af48550-0725-4a4f-adc0-2caabd10d73b" (UID: "5af48550-0725-4a4f-adc0-2caabd10d73b"). InnerVolumeSpecName "kube-api-access-wbcrq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 15:47:57.139474 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:57.139433 2557 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5af48550-0725-4a4f-adc0-2caabd10d73b-config-file\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:47:57.139585 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:57.139484 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbcrq\" (UniqueName: \"kubernetes.io/projected/5af48550-0725-4a4f-adc0-2caabd10d73b-kube-api-access-wbcrq\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:47:57.681143 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:57.681107 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" event={"ID":"5af48550-0725-4a4f-adc0-2caabd10d73b","Type":"ContainerDied","Data":"8d7974a5c7085eb72621419ab0c452cda4cfeb1280b406d895b68cd64acb7243"} Jun 08 15:47:57.681143 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:57.681128 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mgvfq" Jun 08 15:47:57.681394 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:57.681169 2557 scope.go:117] "RemoveContainer" containerID="2623c3fa28cf494b9c456c768fac4af18320cb091e3fd9307baeb56478bace5a" Jun 08 15:47:57.700224 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:57.700200 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mgvfq"] Jun 08 15:47:57.702577 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:57.702557 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mgvfq"] Jun 08 15:47:58.337295 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:58.337255 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5af48550-0725-4a4f-adc0-2caabd10d73b" path="/var/lib/kubelet/pods/5af48550-0725-4a4f-adc0-2caabd10d73b/volumes" Jun 08 15:47:59.242791 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.242754 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:59.342984 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.342955 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:59.676883 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.676856 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/postgres-69fb4594cf-q5xtk"] Jun 08 15:47:59.677207 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.677194 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5af48550-0725-4a4f-adc0-2caabd10d73b" containerName="limitador" Jun 08 15:47:59.677252 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.677209 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5af48550-0725-4a4f-adc0-2caabd10d73b" containerName="limitador" Jun 08 15:47:59.677296 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.677286 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5af48550-0725-4a4f-adc0-2caabd10d73b" containerName="limitador" Jun 08 15:47:59.680098 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.680081 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-q5xtk" Jun 08 15:47:59.682194 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.682172 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jun 08 15:47:59.682336 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.682247 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"default-dockercfg-kggkf\"" Jun 08 15:47:59.682336 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.682288 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"postgres-creds\"" Jun 08 15:47:59.682837 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.682822 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jun 08 15:47:59.687003 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.686979 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/postgres-69fb4594cf-q5xtk"] Jun 08 15:47:59.708016 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.707989 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:47:59.862583 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.862554 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c72b7b96-4c50-4e42-a3e1-8af9a2dff3a2-data\") pod \"postgres-69fb4594cf-q5xtk\" (UID: \"c72b7b96-4c50-4e42-a3e1-8af9a2dff3a2\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-q5xtk" Jun 08 15:47:59.862739 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.862603 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnh4x\" (UniqueName: \"kubernetes.io/projected/c72b7b96-4c50-4e42-a3e1-8af9a2dff3a2-kube-api-access-rnh4x\") pod \"postgres-69fb4594cf-q5xtk\" (UID: \"c72b7b96-4c50-4e42-a3e1-8af9a2dff3a2\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-q5xtk" Jun 08 15:47:59.963907 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.963828 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c72b7b96-4c50-4e42-a3e1-8af9a2dff3a2-data\") pod \"postgres-69fb4594cf-q5xtk\" (UID: \"c72b7b96-4c50-4e42-a3e1-8af9a2dff3a2\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-q5xtk" Jun 08 15:47:59.963907 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.963886 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnh4x\" (UniqueName: \"kubernetes.io/projected/c72b7b96-4c50-4e42-a3e1-8af9a2dff3a2-kube-api-access-rnh4x\") pod \"postgres-69fb4594cf-q5xtk\" (UID: \"c72b7b96-4c50-4e42-a3e1-8af9a2dff3a2\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-q5xtk" Jun 08 15:47:59.964190 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.964172 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c72b7b96-4c50-4e42-a3e1-8af9a2dff3a2-data\") pod \"postgres-69fb4594cf-q5xtk\" (UID: \"c72b7b96-4c50-4e42-a3e1-8af9a2dff3a2\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-q5xtk" Jun 08 15:47:59.971556 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.971534 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnh4x\" (UniqueName: \"kubernetes.io/projected/c72b7b96-4c50-4e42-a3e1-8af9a2dff3a2-kube-api-access-rnh4x\") pod \"postgres-69fb4594cf-q5xtk\" (UID: \"c72b7b96-4c50-4e42-a3e1-8af9a2dff3a2\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-q5xtk" Jun 08 15:47:59.999019 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:47:59.998995 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-q5xtk" Jun 08 15:48:00.124588 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:00.124556 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/postgres-69fb4594cf-q5xtk"] Jun 08 15:48:00.127307 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:48:00.127279 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc72b7b96_4c50_4e42_a3e1_8af9a2dff3a2.slice/crio-6f6985be2c4a1f77eb2425afde20535dc43309d56542576fe3ab17d5d3e0bc28 WatchSource:0}: Error finding container 6f6985be2c4a1f77eb2425afde20535dc43309d56542576fe3ab17d5d3e0bc28: Status 404 returned error can't find the container with id 6f6985be2c4a1f77eb2425afde20535dc43309d56542576fe3ab17d5d3e0bc28 Jun 08 15:48:00.703075 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:00.703042 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-q5xtk" event={"ID":"c72b7b96-4c50-4e42-a3e1-8af9a2dff3a2","Type":"ContainerStarted","Data":"6f6985be2c4a1f77eb2425afde20535dc43309d56542576fe3ab17d5d3e0bc28"} Jun 08 15:48:04.724392 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:04.724357 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-q5xtk" event={"ID":"c72b7b96-4c50-4e42-a3e1-8af9a2dff3a2","Type":"ContainerStarted","Data":"e8e18eb75e83483d01e5841dfbb616240a9fc075ccbe61a309f88e63a3dadbd1"} Jun 08 15:48:04.739729 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:04.739684 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-q5xtk" podStartSLOduration=2.206802892 podStartE2EDuration="5.739667862s" totalImagesPullingTime="3.53286497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:47:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:48:04.738596692 +0000 UTC m=+874.827727743" watchObservedRunningTime="2026-06-08 15:48:04.739667862 +0000 UTC m=+874.828798911" Jun 08 15:48:04.952251 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:04.952212 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:48:05.728679 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:05.728648 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-q5xtk" Jun 08 15:48:11.766095 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:11.766065 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-q5xtk" Jun 08 15:48:12.079802 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:12.079716 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:48:12.181544 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:12.181517 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:48:12.387200 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:12.387174 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:48:30.337512 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:30.337414 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 15:48:30.338504 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:30.338485 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 15:48:30.345113 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:30.345093 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 15:48:30.346034 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:30.346017 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 15:48:35.772085 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:35.772052 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:48:35.871970 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:35.871929 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:48:36.263913 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:36.263876 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 15:48:36.268400 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:36.268384 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 15:48:36.270705 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:36.270675 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 08 15:48:36.270821 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:36.270675 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 08 15:48:36.270821 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:36.270680 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 08 15:48:36.271420 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:36.271406 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-2hssb\"" Jun 08 15:48:36.275136 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:36.275114 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 15:48:36.365225 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:36.365195 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlwc2\" (UniqueName: \"kubernetes.io/projected/72a5fa08-8723-49e1-8650-0798f5a30316-kube-api-access-xlwc2\") pod \"maas-keycloak-0\" (UID: \"72a5fa08-8723-49e1-8650-0798f5a30316\") " pod="keycloak-system/maas-keycloak-0" Jun 08 15:48:36.466192 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:36.466158 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlwc2\" (UniqueName: \"kubernetes.io/projected/72a5fa08-8723-49e1-8650-0798f5a30316-kube-api-access-xlwc2\") pod \"maas-keycloak-0\" (UID: \"72a5fa08-8723-49e1-8650-0798f5a30316\") " pod="keycloak-system/maas-keycloak-0" Jun 08 15:48:36.473527 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:36.473500 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlwc2\" (UniqueName: \"kubernetes.io/projected/72a5fa08-8723-49e1-8650-0798f5a30316-kube-api-access-xlwc2\") pod \"maas-keycloak-0\" (UID: \"72a5fa08-8723-49e1-8650-0798f5a30316\") " pod="keycloak-system/maas-keycloak-0" Jun 08 15:48:36.578782 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:36.578692 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 15:48:36.707637 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:36.707603 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 15:48:36.711406 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:48:36.711371 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72a5fa08_8723_49e1_8650_0798f5a30316.slice/crio-e7eb4edce7800e3aea56fe87a0b2491e14b6d4825ea4d45e8285cda97fde89e7 WatchSource:0}: Error finding container e7eb4edce7800e3aea56fe87a0b2491e14b6d4825ea4d45e8285cda97fde89e7: Status 404 returned error can't find the container with id e7eb4edce7800e3aea56fe87a0b2491e14b6d4825ea4d45e8285cda97fde89e7 Jun 08 15:48:36.863339 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:36.863260 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"72a5fa08-8723-49e1-8650-0798f5a30316","Type":"ContainerStarted","Data":"e7eb4edce7800e3aea56fe87a0b2491e14b6d4825ea4d45e8285cda97fde89e7"} Jun 08 15:48:42.899166 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:42.899125 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"72a5fa08-8723-49e1-8650-0798f5a30316","Type":"ContainerStarted","Data":"03f7336f5f996a6b47b0ecec066d82010c535cba025e1d9540b109862f251720"} Jun 08 15:48:42.918082 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:42.918022 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.607433672 podStartE2EDuration="6.918004918s" totalImagesPullingTime="5.310571246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:48:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:48:42.916328637 +0000 UTC m=+913.005459683" watchObservedRunningTime="2026-06-08 15:48:42.918004918 +0000 UTC m=+913.007135968" Jun 08 15:48:43.579690 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:43.579636 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 08 15:48:43.581725 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:43.581685 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.30:9000/health/started\": dial tcp 10.132.0.30:9000: connect: connection refused" Jun 08 15:48:44.579910 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:44.579869 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.30:9000/health/started\": dial tcp 10.132.0.30:9000: connect: connection refused" Jun 08 15:48:45.579419 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:45.579353 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.30:9000/health/started\": dial tcp 10.132.0.30:9000: connect: connection refused" Jun 08 15:48:46.579603 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:46.579552 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 08 15:48:46.580107 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:46.579869 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.30:9000/health/started\": dial tcp 10.132.0.30:9000: connect: connection refused" Jun 08 15:48:47.580280 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:47.579812 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.30:9000/health/started\": dial tcp 10.132.0.30:9000: connect: connection refused" Jun 08 15:48:48.579496 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:48.579431 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.30:9000/health/started\": dial tcp 10.132.0.30:9000: connect: connection refused" Jun 08 15:48:49.579693 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:49.579643 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.30:9000/health/started\": dial tcp 10.132.0.30:9000: connect: connection refused" Jun 08 15:48:50.579927 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:50.579883 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.30:9000/health/started\": dial tcp 10.132.0.30:9000: connect: connection refused" Jun 08 15:48:51.580029 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:51.579981 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.30:9000/health/started\": dial tcp 10.132.0.30:9000: connect: connection refused" Jun 08 15:48:52.579295 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:52.579241 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.30:9000/health/started\": dial tcp 10.132.0.30:9000: connect: connection refused" Jun 08 15:48:53.579677 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:53.579629 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.30:9000/health/started\": dial tcp 10.132.0.30:9000: connect: connection refused" Jun 08 15:48:54.579583 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:54.579536 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.30:9000/health/started\": dial tcp 10.132.0.30:9000: connect: connection refused" Jun 08 15:48:55.702251 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:55.702201 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 08 15:48:55.722035 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:48:55.721982 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 15:49:05.710386 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:05.710350 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 08 15:49:06.353686 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:06.353651 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:49:06.493738 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:06.493701 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:49:06.821490 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:06.821443 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58574ddfdb-6wmrw"] Jun 08 15:49:06.828672 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:06.828652 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58574ddfdb-6wmrw" Jun 08 15:49:06.831366 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:06.831338 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58574ddfdb-6wmrw"] Jun 08 15:49:06.967678 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:06.967641 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m57l7\" (UniqueName: \"kubernetes.io/projected/a3ad82c2-34b4-4159-9e02-b0b6577df683-kube-api-access-m57l7\") pod \"authorino-58574ddfdb-6wmrw\" (UID: \"a3ad82c2-34b4-4159-9e02-b0b6577df683\") " pod="kuadrant-system/authorino-58574ddfdb-6wmrw" Jun 08 15:49:07.020723 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:07.020684 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58574ddfdb-6wmrw"] Jun 08 15:49:07.020984 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:49:07.020961 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-m57l7], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-58574ddfdb-6wmrw" podUID="a3ad82c2-34b4-4159-9e02-b0b6577df683" Jun 08 15:49:07.037061 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:07.037035 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58574ddfdb-6wmrw" Jun 08 15:49:07.042796 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:07.042775 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58574ddfdb-6wmrw" Jun 08 15:49:07.068600 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:07.068573 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m57l7\" (UniqueName: \"kubernetes.io/projected/a3ad82c2-34b4-4159-9e02-b0b6577df683-kube-api-access-m57l7\") pod \"authorino-58574ddfdb-6wmrw\" (UID: \"a3ad82c2-34b4-4159-9e02-b0b6577df683\") " pod="kuadrant-system/authorino-58574ddfdb-6wmrw" Jun 08 15:49:07.077021 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:07.076964 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m57l7\" (UniqueName: \"kubernetes.io/projected/a3ad82c2-34b4-4159-9e02-b0b6577df683-kube-api-access-m57l7\") pod \"authorino-58574ddfdb-6wmrw\" (UID: \"a3ad82c2-34b4-4159-9e02-b0b6577df683\") " pod="kuadrant-system/authorino-58574ddfdb-6wmrw" Jun 08 15:49:07.169088 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:07.169042 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m57l7\" (UniqueName: \"kubernetes.io/projected/a3ad82c2-34b4-4159-9e02-b0b6577df683-kube-api-access-m57l7\") pod \"a3ad82c2-34b4-4159-9e02-b0b6577df683\" (UID: \"a3ad82c2-34b4-4159-9e02-b0b6577df683\") " Jun 08 15:49:07.171122 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:07.171088 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3ad82c2-34b4-4159-9e02-b0b6577df683-kube-api-access-m57l7" (OuterVolumeSpecName: "kube-api-access-m57l7") pod "a3ad82c2-34b4-4159-9e02-b0b6577df683" (UID: "a3ad82c2-34b4-4159-9e02-b0b6577df683"). InnerVolumeSpecName "kube-api-access-m57l7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 15:49:07.269973 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:07.269940 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m57l7\" (UniqueName: \"kubernetes.io/projected/a3ad82c2-34b4-4159-9e02-b0b6577df683-kube-api-access-m57l7\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:49:08.040869 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:08.040841 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58574ddfdb-6wmrw" Jun 08 15:49:08.067016 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:08.066984 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58574ddfdb-6wmrw"] Jun 08 15:49:08.070784 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:08.070760 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58574ddfdb-6wmrw"] Jun 08 15:49:08.337843 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:08.337760 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3ad82c2-34b4-4159-9e02-b0b6577df683" path="/var/lib/kubelet/pods/a3ad82c2-34b4-4159-9e02-b0b6577df683/volumes" Jun 08 15:49:08.476502 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:08.476447 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-krpmp"] Jun 08 15:49:08.476686 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:08.476650 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-krpmp" podUID="84c45185-4920-4cf9-a754-cd54f53b7ae9" containerName="authorino" containerID="cri-o://42e600e0fb9553a55999f53e42ad7a68338a3d0b36673165e6a9a0161b5e22fd" gracePeriod=30 Jun 08 15:49:08.726248 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:08.726212 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-krpmp" Jun 08 15:49:08.782789 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:08.782755 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2x7q8\" (UniqueName: \"kubernetes.io/projected/84c45185-4920-4cf9-a754-cd54f53b7ae9-kube-api-access-2x7q8\") pod \"84c45185-4920-4cf9-a754-cd54f53b7ae9\" (UID: \"84c45185-4920-4cf9-a754-cd54f53b7ae9\") " Jun 08 15:49:08.784805 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:08.784778 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84c45185-4920-4cf9-a754-cd54f53b7ae9-kube-api-access-2x7q8" (OuterVolumeSpecName: "kube-api-access-2x7q8") pod "84c45185-4920-4cf9-a754-cd54f53b7ae9" (UID: "84c45185-4920-4cf9-a754-cd54f53b7ae9"). InnerVolumeSpecName "kube-api-access-2x7q8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 15:49:08.883535 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:08.883509 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2x7q8\" (UniqueName: \"kubernetes.io/projected/84c45185-4920-4cf9-a754-cd54f53b7ae9-kube-api-access-2x7q8\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:49:09.046878 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:09.046843 2557 generic.go:356] "Generic (PLEG): container finished" podID="84c45185-4920-4cf9-a754-cd54f53b7ae9" containerID="42e600e0fb9553a55999f53e42ad7a68338a3d0b36673165e6a9a0161b5e22fd" exitCode=0 Jun 08 15:49:09.047298 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:09.046885 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-krpmp" Jun 08 15:49:09.047298 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:09.046936 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-krpmp" event={"ID":"84c45185-4920-4cf9-a754-cd54f53b7ae9","Type":"ContainerDied","Data":"42e600e0fb9553a55999f53e42ad7a68338a3d0b36673165e6a9a0161b5e22fd"} Jun 08 15:49:09.047298 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:09.046972 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-krpmp" event={"ID":"84c45185-4920-4cf9-a754-cd54f53b7ae9","Type":"ContainerDied","Data":"d008d2ecd868885b79eb51aa1020cd20fdedf6820e5a436ceb52d2ea73c8b400"} Jun 08 15:49:09.047298 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:09.046990 2557 scope.go:117] "RemoveContainer" containerID="42e600e0fb9553a55999f53e42ad7a68338a3d0b36673165e6a9a0161b5e22fd" Jun 08 15:49:09.055616 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:09.055598 2557 scope.go:117] "RemoveContainer" containerID="42e600e0fb9553a55999f53e42ad7a68338a3d0b36673165e6a9a0161b5e22fd" Jun 08 15:49:09.055872 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:49:09.055854 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42e600e0fb9553a55999f53e42ad7a68338a3d0b36673165e6a9a0161b5e22fd\": container with ID starting with 42e600e0fb9553a55999f53e42ad7a68338a3d0b36673165e6a9a0161b5e22fd not found: ID does not exist" containerID="42e600e0fb9553a55999f53e42ad7a68338a3d0b36673165e6a9a0161b5e22fd" Jun 08 15:49:09.055937 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:09.055880 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42e600e0fb9553a55999f53e42ad7a68338a3d0b36673165e6a9a0161b5e22fd"} err="failed to get container status \"42e600e0fb9553a55999f53e42ad7a68338a3d0b36673165e6a9a0161b5e22fd\": rpc error: code = NotFound desc = could not find container \"42e600e0fb9553a55999f53e42ad7a68338a3d0b36673165e6a9a0161b5e22fd\": container with ID starting with 42e600e0fb9553a55999f53e42ad7a68338a3d0b36673165e6a9a0161b5e22fd not found: ID does not exist" Jun 08 15:49:09.071596 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:09.071569 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-krpmp"] Jun 08 15:49:09.082615 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:09.079319 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-krpmp"] Jun 08 15:49:10.339147 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:10.339112 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84c45185-4920-4cf9-a754-cd54f53b7ae9" path="/var/lib/kubelet/pods/84c45185-4920-4cf9-a754-cd54f53b7ae9/volumes" Jun 08 15:49:11.929767 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:11.929734 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:49:11.948500 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:11.948448 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c5c6b8c85-j4zr6"] Jun 08 15:49:11.948837 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:11.948823 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84c45185-4920-4cf9-a754-cd54f53b7ae9" containerName="authorino" Jun 08 15:49:11.948885 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:11.948839 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="84c45185-4920-4cf9-a754-cd54f53b7ae9" containerName="authorino" Jun 08 15:49:11.948939 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:11.948930 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="84c45185-4920-4cf9-a754-cd54f53b7ae9" containerName="authorino" Jun 08 15:49:11.952089 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:11.952066 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" Jun 08 15:49:11.954299 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:11.954232 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 08 15:49:11.954437 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:11.954422 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 08 15:49:11.954660 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:11.954645 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7gjml\"" Jun 08 15:49:11.960813 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:11.960704 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c5c6b8c85-j4zr6"] Jun 08 15:49:12.010660 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.010626 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgx4w\" (UniqueName: \"kubernetes.io/projected/a4453161-b9de-4f81-aab1-ed986e826edd-kube-api-access-jgx4w\") pod \"maas-controller-5c5c6b8c85-j4zr6\" (UID: \"a4453161-b9de-4f81-aab1-ed986e826edd\") " pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" Jun 08 15:49:12.010815 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.010665 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4453161-b9de-4f81-aab1-ed986e826edd-webhook-cert\") pod \"maas-controller-5c5c6b8c85-j4zr6\" (UID: \"a4453161-b9de-4f81-aab1-ed986e826edd\") " pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" Jun 08 15:49:12.029778 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.029745 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:49:12.088384 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.088348 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:49:12.111714 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.111686 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgx4w\" (UniqueName: \"kubernetes.io/projected/a4453161-b9de-4f81-aab1-ed986e826edd-kube-api-access-jgx4w\") pod \"maas-controller-5c5c6b8c85-j4zr6\" (UID: \"a4453161-b9de-4f81-aab1-ed986e826edd\") " pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" Jun 08 15:49:12.111859 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.111722 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4453161-b9de-4f81-aab1-ed986e826edd-webhook-cert\") pod \"maas-controller-5c5c6b8c85-j4zr6\" (UID: \"a4453161-b9de-4f81-aab1-ed986e826edd\") " pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" Jun 08 15:49:12.114127 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.114099 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4453161-b9de-4f81-aab1-ed986e826edd-webhook-cert\") pod \"maas-controller-5c5c6b8c85-j4zr6\" (UID: \"a4453161-b9de-4f81-aab1-ed986e826edd\") " pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" Jun 08 15:49:12.118794 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.118772 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6454c6fdb-gpk4j"] Jun 08 15:49:12.121111 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.121089 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6454c6fdb-gpk4j" Jun 08 15:49:12.123635 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.123613 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgx4w\" (UniqueName: \"kubernetes.io/projected/a4453161-b9de-4f81-aab1-ed986e826edd-kube-api-access-jgx4w\") pod \"maas-controller-5c5c6b8c85-j4zr6\" (UID: \"a4453161-b9de-4f81-aab1-ed986e826edd\") " pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" Jun 08 15:49:12.130102 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.130083 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6454c6fdb-gpk4j"] Jun 08 15:49:12.212889 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.212816 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5bae33f1-3a8c-4623-b170-fcc5da1031fb-webhook-cert\") pod \"maas-controller-6454c6fdb-gpk4j\" (UID: \"5bae33f1-3a8c-4623-b170-fcc5da1031fb\") " pod="opendatahub/maas-controller-6454c6fdb-gpk4j" Jun 08 15:49:12.212889 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.212848 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t5sz\" (UniqueName: \"kubernetes.io/projected/5bae33f1-3a8c-4623-b170-fcc5da1031fb-kube-api-access-2t5sz\") pod \"maas-controller-6454c6fdb-gpk4j\" (UID: \"5bae33f1-3a8c-4623-b170-fcc5da1031fb\") " pod="opendatahub/maas-controller-6454c6fdb-gpk4j" Jun 08 15:49:12.265102 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.265070 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" Jun 08 15:49:12.314358 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.314327 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5bae33f1-3a8c-4623-b170-fcc5da1031fb-webhook-cert\") pod \"maas-controller-6454c6fdb-gpk4j\" (UID: \"5bae33f1-3a8c-4623-b170-fcc5da1031fb\") " pod="opendatahub/maas-controller-6454c6fdb-gpk4j" Jun 08 15:49:12.314448 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.314378 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t5sz\" (UniqueName: \"kubernetes.io/projected/5bae33f1-3a8c-4623-b170-fcc5da1031fb-kube-api-access-2t5sz\") pod \"maas-controller-6454c6fdb-gpk4j\" (UID: \"5bae33f1-3a8c-4623-b170-fcc5da1031fb\") " pod="opendatahub/maas-controller-6454c6fdb-gpk4j" Jun 08 15:49:12.317715 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.317684 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5bae33f1-3a8c-4623-b170-fcc5da1031fb-webhook-cert\") pod \"maas-controller-6454c6fdb-gpk4j\" (UID: \"5bae33f1-3a8c-4623-b170-fcc5da1031fb\") " pod="opendatahub/maas-controller-6454c6fdb-gpk4j" Jun 08 15:49:12.322633 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.322606 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t5sz\" (UniqueName: \"kubernetes.io/projected/5bae33f1-3a8c-4623-b170-fcc5da1031fb-kube-api-access-2t5sz\") pod \"maas-controller-6454c6fdb-gpk4j\" (UID: \"5bae33f1-3a8c-4623-b170-fcc5da1031fb\") " pod="opendatahub/maas-controller-6454c6fdb-gpk4j" Jun 08 15:49:12.399918 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.399895 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c5c6b8c85-j4zr6"] Jun 08 15:49:12.401974 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:49:12.401937 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4453161_b9de_4f81_aab1_ed986e826edd.slice/crio-dea1198040533bcf4e03089b3c8c6f105011af7d7f5e5c819ffab2eebf9e81ed WatchSource:0}: Error finding container dea1198040533bcf4e03089b3c8c6f105011af7d7f5e5c819ffab2eebf9e81ed: Status 404 returned error can't find the container with id dea1198040533bcf4e03089b3c8c6f105011af7d7f5e5c819ffab2eebf9e81ed Jun 08 15:49:12.440444 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.440411 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6454c6fdb-gpk4j" Jun 08 15:49:12.564415 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:12.564385 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6454c6fdb-gpk4j"] Jun 08 15:49:12.567809 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:49:12.567784 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bae33f1_3a8c_4623_b170_fcc5da1031fb.slice/crio-36a227bcbb0bda37cd4ec5710507623f66e34ee0049e69fb6aa3b3eaadd38541 WatchSource:0}: Error finding container 36a227bcbb0bda37cd4ec5710507623f66e34ee0049e69fb6aa3b3eaadd38541: Status 404 returned error can't find the container with id 36a227bcbb0bda37cd4ec5710507623f66e34ee0049e69fb6aa3b3eaadd38541 Jun 08 15:49:13.069215 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:13.069180 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" event={"ID":"a4453161-b9de-4f81-aab1-ed986e826edd","Type":"ContainerStarted","Data":"dea1198040533bcf4e03089b3c8c6f105011af7d7f5e5c819ffab2eebf9e81ed"} Jun 08 15:49:13.070653 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:13.070617 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6454c6fdb-gpk4j" event={"ID":"5bae33f1-3a8c-4623-b170-fcc5da1031fb","Type":"ContainerStarted","Data":"36a227bcbb0bda37cd4ec5710507623f66e34ee0049e69fb6aa3b3eaadd38541"} Jun 08 15:49:15.265234 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:15.265198 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:49:15.363800 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:15.363694 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:49:16.094478 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:16.094439 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" event={"ID":"a4453161-b9de-4f81-aab1-ed986e826edd","Type":"ContainerStarted","Data":"203c908cce21359e994feee48c92f3694110349f2bb56d0ed17211c88f756056"} Jun 08 15:49:16.094681 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:16.094647 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" Jun 08 15:49:16.095902 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:16.095881 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6454c6fdb-gpk4j" event={"ID":"5bae33f1-3a8c-4623-b170-fcc5da1031fb","Type":"ContainerStarted","Data":"d2fe390687475fd5d0af29e88df435bac74e0d767bfe8816879af16670d022d3"} Jun 08 15:49:16.096033 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:16.096008 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6454c6fdb-gpk4j" Jun 08 15:49:16.112097 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:16.112054 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" podStartSLOduration=2.455845847 podStartE2EDuration="5.112042646s" totalImagesPullingTime="2.656196799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:49:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:49:16.110208728 +0000 UTC m=+946.199339789" watchObservedRunningTime="2026-06-08 15:49:16.112042646 +0000 UTC m=+946.201173696" Jun 08 15:49:16.126322 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:16.126280 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6454c6fdb-gpk4j" podStartSLOduration=1.625850907 podStartE2EDuration="4.12625806s" totalImagesPullingTime="2.500407153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:49:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:49:16.125545778 +0000 UTC m=+946.214676828" watchObservedRunningTime="2026-06-08 15:49:16.12625806 +0000 UTC m=+946.215389154" Jun 08 15:49:27.106272 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:27.106240 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" Jun 08 15:49:27.106687 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:27.106308 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6454c6fdb-gpk4j" Jun 08 15:49:27.166212 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:27.166180 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c5c6b8c85-j4zr6"] Jun 08 15:49:27.166436 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:27.166393 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" podUID="a4453161-b9de-4f81-aab1-ed986e826edd" containerName="manager" containerID="cri-o://203c908cce21359e994feee48c92f3694110349f2bb56d0ed17211c88f756056" gracePeriod=30 Jun 08 15:49:27.408183 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:27.408154 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" Jun 08 15:49:27.549250 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:27.549206 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4453161-b9de-4f81-aab1-ed986e826edd-webhook-cert\") pod \"a4453161-b9de-4f81-aab1-ed986e826edd\" (UID: \"a4453161-b9de-4f81-aab1-ed986e826edd\") " Jun 08 15:49:27.549491 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:27.549281 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgx4w\" (UniqueName: \"kubernetes.io/projected/a4453161-b9de-4f81-aab1-ed986e826edd-kube-api-access-jgx4w\") pod \"a4453161-b9de-4f81-aab1-ed986e826edd\" (UID: \"a4453161-b9de-4f81-aab1-ed986e826edd\") " Jun 08 15:49:27.551507 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:27.551450 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4453161-b9de-4f81-aab1-ed986e826edd-kube-api-access-jgx4w" (OuterVolumeSpecName: "kube-api-access-jgx4w") pod "a4453161-b9de-4f81-aab1-ed986e826edd" (UID: "a4453161-b9de-4f81-aab1-ed986e826edd"). InnerVolumeSpecName "kube-api-access-jgx4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 15:49:27.551621 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:27.551540 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4453161-b9de-4f81-aab1-ed986e826edd-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a4453161-b9de-4f81-aab1-ed986e826edd" (UID: "a4453161-b9de-4f81-aab1-ed986e826edd"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 15:49:27.650776 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:27.650690 2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4453161-b9de-4f81-aab1-ed986e826edd-webhook-cert\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:49:27.650776 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:27.650721 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgx4w\" (UniqueName: \"kubernetes.io/projected/a4453161-b9de-4f81-aab1-ed986e826edd-kube-api-access-jgx4w\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:49:27.893763 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:27.893726 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:49:27.994147 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:27.994064 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:49:28.151717 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:28.151680 2557 generic.go:356] "Generic (PLEG): container finished" podID="a4453161-b9de-4f81-aab1-ed986e826edd" containerID="203c908cce21359e994feee48c92f3694110349f2bb56d0ed17211c88f756056" exitCode=0 Jun 08 15:49:28.152124 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:28.151721 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" Jun 08 15:49:28.152124 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:28.151746 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" event={"ID":"a4453161-b9de-4f81-aab1-ed986e826edd","Type":"ContainerDied","Data":"203c908cce21359e994feee48c92f3694110349f2bb56d0ed17211c88f756056"} Jun 08 15:49:28.152124 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:28.151814 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5c6b8c85-j4zr6" event={"ID":"a4453161-b9de-4f81-aab1-ed986e826edd","Type":"ContainerDied","Data":"dea1198040533bcf4e03089b3c8c6f105011af7d7f5e5c819ffab2eebf9e81ed"} Jun 08 15:49:28.152124 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:28.151839 2557 scope.go:117] "RemoveContainer" containerID="203c908cce21359e994feee48c92f3694110349f2bb56d0ed17211c88f756056" Jun 08 15:49:28.160270 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:28.160252 2557 scope.go:117] "RemoveContainer" containerID="203c908cce21359e994feee48c92f3694110349f2bb56d0ed17211c88f756056" Jun 08 15:49:28.160528 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:49:28.160506 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"203c908cce21359e994feee48c92f3694110349f2bb56d0ed17211c88f756056\": container with ID starting with 203c908cce21359e994feee48c92f3694110349f2bb56d0ed17211c88f756056 not found: ID does not exist" containerID="203c908cce21359e994feee48c92f3694110349f2bb56d0ed17211c88f756056" Jun 08 15:49:28.160596 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:28.160536 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"203c908cce21359e994feee48c92f3694110349f2bb56d0ed17211c88f756056"} err="failed to get container status \"203c908cce21359e994feee48c92f3694110349f2bb56d0ed17211c88f756056\": rpc error: code = NotFound desc = could not find container \"203c908cce21359e994feee48c92f3694110349f2bb56d0ed17211c88f756056\": container with ID starting with 203c908cce21359e994feee48c92f3694110349f2bb56d0ed17211c88f756056 not found: ID does not exist" Jun 08 15:49:28.173694 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:28.173667 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c5c6b8c85-j4zr6"] Jun 08 15:49:28.177728 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:28.177705 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5c5c6b8c85-j4zr6"] Jun 08 15:49:28.337645 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:28.337561 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4453161-b9de-4f81-aab1-ed986e826edd" path="/var/lib/kubelet/pods/a4453161-b9de-4f81-aab1-ed986e826edd/volumes" Jun 08 15:49:44.321054 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:44.320969 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:49:44.396136 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:44.396100 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/maas-api-658d88596d-qnm79"] Jun 08 15:49:44.396541 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:44.396522 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4453161-b9de-4f81-aab1-ed986e826edd" containerName="manager" Jun 08 15:49:44.396691 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:44.396545 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4453161-b9de-4f81-aab1-ed986e826edd" containerName="manager" Jun 08 15:49:44.396691 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:44.396641 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4453161-b9de-4f81-aab1-ed986e826edd" containerName="manager" Jun 08 15:49:44.401318 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:44.401296 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-658d88596d-qnm79" Jun 08 15:49:44.403650 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:44.403633 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"maas-api-dockercfg-sdjvj\"" Jun 08 15:49:44.403771 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:44.403660 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jun 08 15:49:44.409491 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:44.409453 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-658d88596d-qnm79"] Jun 08 15:49:44.492007 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:44.491971 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56-maas-api-tls\") pod \"maas-api-658d88596d-qnm79\" (UID: \"95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56\") " pod="redhat-ai-gateway-infra/maas-api-658d88596d-qnm79" Jun 08 15:49:44.492193 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:44.492092 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmv4p\" (UniqueName: \"kubernetes.io/projected/95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56-kube-api-access-tmv4p\") pod \"maas-api-658d88596d-qnm79\" (UID: \"95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56\") " pod="redhat-ai-gateway-infra/maas-api-658d88596d-qnm79" Jun 08 15:49:44.592824 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:44.592734 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmv4p\" (UniqueName: \"kubernetes.io/projected/95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56-kube-api-access-tmv4p\") pod \"maas-api-658d88596d-qnm79\" (UID: \"95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56\") " pod="redhat-ai-gateway-infra/maas-api-658d88596d-qnm79" Jun 08 15:49:44.592824 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:44.592797 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56-maas-api-tls\") pod \"maas-api-658d88596d-qnm79\" (UID: \"95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56\") " pod="redhat-ai-gateway-infra/maas-api-658d88596d-qnm79" Jun 08 15:49:44.593042 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:49:44.592890 2557 secret.go:189] Couldn't get secret redhat-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jun 08 15:49:44.593042 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:49:44.592957 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56-maas-api-tls podName:95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56 nodeName:}" failed. No retries permitted until 2026-06-08 15:49:45.092941543 +0000 UTC m=+975.182072570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56-maas-api-tls") pod "maas-api-658d88596d-qnm79" (UID: "95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56") : secret "maas-api-serving-cert" not found Jun 08 15:49:44.603227 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:44.603201 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmv4p\" (UniqueName: \"kubernetes.io/projected/95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56-kube-api-access-tmv4p\") pod \"maas-api-658d88596d-qnm79\" (UID: \"95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56\") " pod="redhat-ai-gateway-infra/maas-api-658d88596d-qnm79" Jun 08 15:49:44.824529 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:44.824492 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:49:44.926628 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:44.926597 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:49:45.016913 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.016883 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:49:45.020487 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.020448 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-89lhg"] Jun 08 15:49:45.066602 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.066577 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-89lhg"] Jun 08 15:49:45.066714 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.066661 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-89lhg" Jun 08 15:49:45.068761 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.068741 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 08 15:49:45.068879 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.068805 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-wpmr7\"" Jun 08 15:49:45.091231 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.091190 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-m42t7"] Jun 08 15:49:45.098281 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.098260 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d425b\" (UniqueName: \"kubernetes.io/projected/48b98f5c-6b3b-44ab-a57f-1d77493c665c-kube-api-access-d425b\") pod \"payload-pre-processing-67d7689b65-89lhg\" (UID: \"48b98f5c-6b3b-44ab-a57f-1d77493c665c\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-89lhg" Jun 08 15:49:45.098394 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.098361 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56-maas-api-tls\") pod \"maas-api-658d88596d-qnm79\" (UID: \"95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56\") " pod="redhat-ai-gateway-infra/maas-api-658d88596d-qnm79" Jun 08 15:49:45.098668 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.098648 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-m42t7" Jun 08 15:49:45.100883 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.100863 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56-maas-api-tls\") pod \"maas-api-658d88596d-qnm79\" (UID: \"95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56\") " pod="redhat-ai-gateway-infra/maas-api-658d88596d-qnm79" Jun 08 15:49:45.104979 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.104959 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-m42t7"] Jun 08 15:49:45.198984 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.198878 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qcp7\" (UniqueName: \"kubernetes.io/projected/06d84685-b8db-4564-a09a-da6ae5cc1017-kube-api-access-9qcp7\") pod \"payload-processing-7c6c859784-m42t7\" (UID: \"06d84685-b8db-4564-a09a-da6ae5cc1017\") " pod="openshift-ingress/payload-processing-7c6c859784-m42t7" Jun 08 15:49:45.198984 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.198936 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d425b\" (UniqueName: \"kubernetes.io/projected/48b98f5c-6b3b-44ab-a57f-1d77493c665c-kube-api-access-d425b\") pod \"payload-pre-processing-67d7689b65-89lhg\" (UID: \"48b98f5c-6b3b-44ab-a57f-1d77493c665c\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-89lhg" Jun 08 15:49:45.209274 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.209243 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d425b\" (UniqueName: \"kubernetes.io/projected/48b98f5c-6b3b-44ab-a57f-1d77493c665c-kube-api-access-d425b\") pod \"payload-pre-processing-67d7689b65-89lhg\" (UID: \"48b98f5c-6b3b-44ab-a57f-1d77493c665c\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-89lhg" Jun 08 15:49:45.299632 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.299601 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qcp7\" (UniqueName: \"kubernetes.io/projected/06d84685-b8db-4564-a09a-da6ae5cc1017-kube-api-access-9qcp7\") pod \"payload-processing-7c6c859784-m42t7\" (UID: \"06d84685-b8db-4564-a09a-da6ae5cc1017\") " pod="openshift-ingress/payload-processing-7c6c859784-m42t7" Jun 08 15:49:45.306828 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.306802 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qcp7\" (UniqueName: \"kubernetes.io/projected/06d84685-b8db-4564-a09a-da6ae5cc1017-kube-api-access-9qcp7\") pod \"payload-processing-7c6c859784-m42t7\" (UID: \"06d84685-b8db-4564-a09a-da6ae5cc1017\") " pod="openshift-ingress/payload-processing-7c6c859784-m42t7" Jun 08 15:49:45.311581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.311559 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-658d88596d-qnm79" Jun 08 15:49:45.377154 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.377126 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-89lhg" Jun 08 15:49:45.417867 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.417600 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-m42t7" Jun 08 15:49:45.455235 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.455051 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-658d88596d-qnm79"] Jun 08 15:49:45.458247 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:49:45.458218 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95f4a1c9_ca25_4d1b_a14a_56c3bc1aac56.slice/crio-77ad573b706c50e85214f970fde7072d9d1009c1df793ba0dc711300b697da23 WatchSource:0}: Error finding container 77ad573b706c50e85214f970fde7072d9d1009c1df793ba0dc711300b697da23: Status 404 returned error can't find the container with id 77ad573b706c50e85214f970fde7072d9d1009c1df793ba0dc711300b697da23 Jun 08 15:49:45.460114 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.460096 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 15:49:45.524938 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.524907 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-89lhg"] Jun 08 15:49:45.530513 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:49:45.527175 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48b98f5c_6b3b_44ab_a57f_1d77493c665c.slice/crio-646c255063152284db086ee8b22ff9d6ca514e95b94f60b77a3c72a171effd9e WatchSource:0}: Error finding container 646c255063152284db086ee8b22ff9d6ca514e95b94f60b77a3c72a171effd9e: Status 404 returned error can't find the container with id 646c255063152284db086ee8b22ff9d6ca514e95b94f60b77a3c72a171effd9e Jun 08 15:49:45.569509 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:45.569371 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-m42t7"] Jun 08 15:49:45.572100 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:49:45.572076 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06d84685_b8db_4564_a09a_da6ae5cc1017.slice/crio-15ce3a016a60ed356d306026c5f2830cc45e214c2a2ab81db862761fdba1fb88 WatchSource:0}: Error finding container 15ce3a016a60ed356d306026c5f2830cc45e214c2a2ab81db862761fdba1fb88: Status 404 returned error can't find the container with id 15ce3a016a60ed356d306026c5f2830cc45e214c2a2ab81db862761fdba1fb88 Jun 08 15:49:46.234721 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:46.234681 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-658d88596d-qnm79" event={"ID":"95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56","Type":"ContainerStarted","Data":"77ad573b706c50e85214f970fde7072d9d1009c1df793ba0dc711300b697da23"} Jun 08 15:49:46.236933 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:46.236889 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-89lhg" event={"ID":"48b98f5c-6b3b-44ab-a57f-1d77493c665c","Type":"ContainerStarted","Data":"646c255063152284db086ee8b22ff9d6ca514e95b94f60b77a3c72a171effd9e"} Jun 08 15:49:46.239332 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:46.239288 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-m42t7" event={"ID":"06d84685-b8db-4564-a09a-da6ae5cc1017","Type":"ContainerStarted","Data":"15ce3a016a60ed356d306026c5f2830cc45e214c2a2ab81db862761fdba1fb88"} Jun 08 15:49:48.250155 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:48.250116 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-658d88596d-qnm79" event={"ID":"95f4a1c9-ca25-4d1b-a14a-56c3bc1aac56","Type":"ContainerStarted","Data":"c0212722728d64726b5d9a4cd4877aace7cd306bc1023cde4a20a00a06829827"} Jun 08 15:49:48.250648 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:48.250214 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/maas-api-658d88596d-qnm79" Jun 08 15:49:48.251880 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:48.251860 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-89lhg" event={"ID":"48b98f5c-6b3b-44ab-a57f-1d77493c665c","Type":"ContainerStarted","Data":"6ccac6dcd0aa3ff831ac3814118bf3a7fd55970eb169891f80bda318ca11ad79"} Jun 08 15:49:48.252012 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:48.251914 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-89lhg" Jun 08 15:49:48.253435 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:48.253403 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-m42t7" event={"ID":"06d84685-b8db-4564-a09a-da6ae5cc1017","Type":"ContainerStarted","Data":"567a94d038198ed46b59cbd7cc5482a26d626acc963a1d97ff6e0d3f230db514"} Jun 08 15:49:48.266312 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:48.266272 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/maas-api-658d88596d-qnm79" podStartSLOduration=2.5247898859999998 podStartE2EDuration="4.266258488s" totalImagesPullingTime="1.741468602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:49:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:49:48.265217549 +0000 UTC m=+978.354348600" watchObservedRunningTime="2026-06-08 15:49:48.266258488 +0000 UTC m=+978.355389537" Jun 08 15:49:48.282680 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:48.282641 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7c6c859784-m42t7" podStartSLOduration=1.652028008 podStartE2EDuration="3.282630382s" totalImagesPullingTime="1.630602374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:49:48.280626466 +0000 UTC m=+978.369757532" watchObservedRunningTime="2026-06-08 15:49:48.282630382 +0000 UTC m=+978.371761432" Jun 08 15:49:48.294672 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:48.294631 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-67d7689b65-89lhg" podStartSLOduration=1.623528699 podStartE2EDuration="3.294617767s" totalImagesPullingTime="1.671089068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:49:48.293752461 +0000 UTC m=+978.382883521" watchObservedRunningTime="2026-06-08 15:49:48.294617767 +0000 UTC m=+978.383748816" Jun 08 15:49:49.259183 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:49.259156 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-m42t7" Jun 08 15:49:54.264545 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:54.264516 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ai-gateway-infra/maas-api-658d88596d-qnm79" Jun 08 15:49:55.981593 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:55.981560 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 15:49:55.981985 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:55.981773 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" containerID="cri-o://03f7336f5f996a6b47b0ecec066d82010c535cba025e1d9540b109862f251720" gracePeriod=30 Jun 08 15:49:58.019840 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.019816 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 15:49:58.109841 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.109802 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlwc2\" (UniqueName: \"kubernetes.io/projected/72a5fa08-8723-49e1-8650-0798f5a30316-kube-api-access-xlwc2\") pod \"72a5fa08-8723-49e1-8650-0798f5a30316\" (UID: \"72a5fa08-8723-49e1-8650-0798f5a30316\") " Jun 08 15:49:58.112034 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.112004 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72a5fa08-8723-49e1-8650-0798f5a30316-kube-api-access-xlwc2" (OuterVolumeSpecName: "kube-api-access-xlwc2") pod "72a5fa08-8723-49e1-8650-0798f5a30316" (UID: "72a5fa08-8723-49e1-8650-0798f5a30316"). InnerVolumeSpecName "kube-api-access-xlwc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 15:49:58.211458 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.211370 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlwc2\" (UniqueName: \"kubernetes.io/projected/72a5fa08-8723-49e1-8650-0798f5a30316-kube-api-access-xlwc2\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:49:58.302594 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.302555 2557 generic.go:356] "Generic (PLEG): container finished" podID="72a5fa08-8723-49e1-8650-0798f5a30316" containerID="03f7336f5f996a6b47b0ecec066d82010c535cba025e1d9540b109862f251720" exitCode=143 Jun 08 15:49:58.302774 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.302606 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 15:49:58.302774 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.302643 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"72a5fa08-8723-49e1-8650-0798f5a30316","Type":"ContainerDied","Data":"03f7336f5f996a6b47b0ecec066d82010c535cba025e1d9540b109862f251720"} Jun 08 15:49:58.302774 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.302700 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"72a5fa08-8723-49e1-8650-0798f5a30316","Type":"ContainerDied","Data":"e7eb4edce7800e3aea56fe87a0b2491e14b6d4825ea4d45e8285cda97fde89e7"} Jun 08 15:49:58.302774 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.302726 2557 scope.go:117] "RemoveContainer" containerID="03f7336f5f996a6b47b0ecec066d82010c535cba025e1d9540b109862f251720" Jun 08 15:49:58.312928 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.312912 2557 scope.go:117] "RemoveContainer" containerID="03f7336f5f996a6b47b0ecec066d82010c535cba025e1d9540b109862f251720" Jun 08 15:49:58.313174 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:49:58.313149 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03f7336f5f996a6b47b0ecec066d82010c535cba025e1d9540b109862f251720\": container with ID starting with 03f7336f5f996a6b47b0ecec066d82010c535cba025e1d9540b109862f251720 not found: ID does not exist" containerID="03f7336f5f996a6b47b0ecec066d82010c535cba025e1d9540b109862f251720" Jun 08 15:49:58.313223 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.313184 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03f7336f5f996a6b47b0ecec066d82010c535cba025e1d9540b109862f251720"} err="failed to get container status \"03f7336f5f996a6b47b0ecec066d82010c535cba025e1d9540b109862f251720\": rpc error: code = NotFound desc = could not find container \"03f7336f5f996a6b47b0ecec066d82010c535cba025e1d9540b109862f251720\": container with ID starting with 03f7336f5f996a6b47b0ecec066d82010c535cba025e1d9540b109862f251720 not found: ID does not exist" Jun 08 15:49:58.322309 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.322284 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 15:49:58.324582 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.324563 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 15:49:58.337232 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.337213 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" path="/var/lib/kubelet/pods/72a5fa08-8723-49e1-8650-0798f5a30316/volumes" Jun 08 15:49:58.346211 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.346192 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 15:49:58.346562 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.346548 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" Jun 08 15:49:58.346619 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.346563 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" Jun 08 15:49:58.346657 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.346647 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="72a5fa08-8723-49e1-8650-0798f5a30316" containerName="keycloak" Jun 08 15:49:58.351179 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.351164 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 15:49:58.353346 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.353328 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-2hssb\"" Jun 08 15:49:58.353456 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.353331 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 08 15:49:58.353456 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.353331 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 08 15:49:58.353456 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.353330 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 08 15:49:58.353456 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.353334 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 08 15:49:58.361667 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.361647 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 15:49:58.413224 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.413192 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/9073ac70-6edc-4d72-9e61-510264788b95-test-realms\") pod \"maas-keycloak-0\" (UID: \"9073ac70-6edc-4d72-9e61-510264788b95\") " pod="keycloak-system/maas-keycloak-0" Jun 08 15:49:58.413363 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.413250 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvzbg\" (UniqueName: \"kubernetes.io/projected/9073ac70-6edc-4d72-9e61-510264788b95-kube-api-access-qvzbg\") pod \"maas-keycloak-0\" (UID: \"9073ac70-6edc-4d72-9e61-510264788b95\") " pod="keycloak-system/maas-keycloak-0" Jun 08 15:49:58.509950 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.509859 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:49:58.513932 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.513912 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvzbg\" (UniqueName: \"kubernetes.io/projected/9073ac70-6edc-4d72-9e61-510264788b95-kube-api-access-qvzbg\") pod \"maas-keycloak-0\" (UID: \"9073ac70-6edc-4d72-9e61-510264788b95\") " pod="keycloak-system/maas-keycloak-0" Jun 08 15:49:58.514026 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.513991 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/9073ac70-6edc-4d72-9e61-510264788b95-test-realms\") pod \"maas-keycloak-0\" (UID: \"9073ac70-6edc-4d72-9e61-510264788b95\") " pod="keycloak-system/maas-keycloak-0" Jun 08 15:49:58.514740 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.514722 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/9073ac70-6edc-4d72-9e61-510264788b95-test-realms\") pod \"maas-keycloak-0\" (UID: \"9073ac70-6edc-4d72-9e61-510264788b95\") " pod="keycloak-system/maas-keycloak-0" Jun 08 15:49:58.522003 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.521982 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvzbg\" (UniqueName: \"kubernetes.io/projected/9073ac70-6edc-4d72-9e61-510264788b95-kube-api-access-qvzbg\") pod \"maas-keycloak-0\" (UID: \"9073ac70-6edc-4d72-9e61-510264788b95\") " pod="keycloak-system/maas-keycloak-0" Jun 08 15:49:58.558298 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.558264 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:49:58.661720 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.661694 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 08 15:49:58.789291 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:58.789232 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 08 15:49:58.792151 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:49:58.792122 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9073ac70_6edc_4d72_9e61_510264788b95.slice/crio-bfd5ea213ae92b796769f6ccd7873dff5192158fe8c87d3e22580a79e4314397 WatchSource:0}: Error finding container bfd5ea213ae92b796769f6ccd7873dff5192158fe8c87d3e22580a79e4314397: Status 404 returned error can't find the container with id bfd5ea213ae92b796769f6ccd7873dff5192158fe8c87d3e22580a79e4314397 Jun 08 15:49:59.262015 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:59.261962 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-89lhg" Jun 08 15:49:59.311429 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:59.311391 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"9073ac70-6edc-4d72-9e61-510264788b95","Type":"ContainerStarted","Data":"e37e9fe1dee1471193cee2f8b2f7a2a761b0f4c478fd3acdf1e74ac37ec8d084"} Jun 08 15:49:59.311645 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:59.311476 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"9073ac70-6edc-4d72-9e61-510264788b95","Type":"ContainerStarted","Data":"bfd5ea213ae92b796769f6ccd7873dff5192158fe8c87d3e22580a79e4314397"} Jun 08 15:49:59.330310 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:59.330201 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.052622622 podStartE2EDuration="1.330179838s" totalImagesPullingTime="277.557216ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:49:59.327098241 +0000 UTC m=+989.416229299" watchObservedRunningTime="2026-06-08 15:49:59.330179838 +0000 UTC m=+989.419310901" Jun 08 15:49:59.662789 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:59.662746 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 08 15:49:59.664580 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:49:59.664544 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9073ac70-6edc-4d72-9e61-510264788b95" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.37:9000/health/started\": dial tcp 10.132.0.37:9000: connect: connection refused" Jun 08 15:50:00.266717 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:00.266683 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-m42t7" Jun 08 15:50:00.662488 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:00.662413 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9073ac70-6edc-4d72-9e61-510264788b95" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.37:9000/health/started\": dial tcp 10.132.0.37:9000: connect: connection refused" Jun 08 15:50:01.662297 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:01.662251 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9073ac70-6edc-4d72-9e61-510264788b95" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.37:9000/health/started\": dial tcp 10.132.0.37:9000: connect: connection refused" Jun 08 15:50:02.662392 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:02.662340 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9073ac70-6edc-4d72-9e61-510264788b95" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.37:9000/health/started\": dial tcp 10.132.0.37:9000: connect: connection refused" Jun 08 15:50:03.663057 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:03.663000 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9073ac70-6edc-4d72-9e61-510264788b95" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.37:9000/health/started\": dial tcp 10.132.0.37:9000: connect: connection refused" Jun 08 15:50:04.662741 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:04.662696 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9073ac70-6edc-4d72-9e61-510264788b95" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.37:9000/health/started\": dial tcp 10.132.0.37:9000: connect: connection refused" Jun 08 15:50:05.663215 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:05.663155 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9073ac70-6edc-4d72-9e61-510264788b95" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.37:9000/health/started\": dial tcp 10.132.0.37:9000: connect: connection refused" Jun 08 15:50:06.663053 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:06.662994 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9073ac70-6edc-4d72-9e61-510264788b95" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.37:9000/health/started\": dial tcp 10.132.0.37:9000: connect: connection refused" Jun 08 15:50:07.663117 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:07.663072 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9073ac70-6edc-4d72-9e61-510264788b95" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.37:9000/health/started\": dial tcp 10.132.0.37:9000: connect: connection refused" Jun 08 15:50:08.662646 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:08.662600 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 08 15:50:08.663005 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:08.662963 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9073ac70-6edc-4d72-9e61-510264788b95" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.37:9000/health/started\": dial tcp 10.132.0.37:9000: connect: connection refused" Jun 08 15:50:09.662418 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:09.662369 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9073ac70-6edc-4d72-9e61-510264788b95" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.37:9000/health/started\": dial tcp 10.132.0.37:9000: connect: connection refused" Jun 08 15:50:10.662249 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:10.662201 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9073ac70-6edc-4d72-9e61-510264788b95" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.37:9000/health/started\": dial tcp 10.132.0.37:9000: connect: connection refused" Jun 08 15:50:11.662160 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:11.662111 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9073ac70-6edc-4d72-9e61-510264788b95" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.37:9000/health/started\": dial tcp 10.132.0.37:9000: connect: connection refused" Jun 08 15:50:12.754083 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:12.754042 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 08 15:50:12.768976 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:12.768940 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="9073ac70-6edc-4d72-9e61-510264788b95" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 15:50:22.759898 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:22.759866 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 08 15:50:35.329018 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:35.328980 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:50:35.431500 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:35.431424 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:50:40.954695 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:40.954663 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:50:44.441120 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:44.441083 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:50:44.492766 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:44.492737 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:50:49.728843 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:50:49.728812 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:51:07.845004 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:07.844971 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:51:12.422404 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.422368 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h"] Jun 08 15:51:12.426316 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.426297 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.429498 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.429475 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 08 15:51:12.429658 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.429563 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 08 15:51:12.429775 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.429563 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ngk5s\"" Jun 08 15:51:12.429775 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.429563 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 08 15:51:12.435148 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.435124 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h"] Jun 08 15:51:12.457925 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.457900 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:51:12.487057 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.487030 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f714467d-5b58-497d-918a-eddc34b42a0d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.487163 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.487062 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv6p6\" (UniqueName: \"kubernetes.io/projected/f714467d-5b58-497d-918a-eddc34b42a0d-kube-api-access-hv6p6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.487163 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.487131 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f714467d-5b58-497d-918a-eddc34b42a0d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.487240 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.487157 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f714467d-5b58-497d-918a-eddc34b42a0d-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.487240 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.487210 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f714467d-5b58-497d-918a-eddc34b42a0d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.487305 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.487248 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f714467d-5b58-497d-918a-eddc34b42a0d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.588456 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.588429 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f714467d-5b58-497d-918a-eddc34b42a0d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.588627 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.588480 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f714467d-5b58-497d-918a-eddc34b42a0d-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.588627 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.588508 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f714467d-5b58-497d-918a-eddc34b42a0d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.588627 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.588532 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f714467d-5b58-497d-918a-eddc34b42a0d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.588627 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.588580 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f714467d-5b58-497d-918a-eddc34b42a0d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.588627 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.588598 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv6p6\" (UniqueName: \"kubernetes.io/projected/f714467d-5b58-497d-918a-eddc34b42a0d-kube-api-access-hv6p6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.588990 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.588967 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f714467d-5b58-497d-918a-eddc34b42a0d-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.589057 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.589017 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f714467d-5b58-497d-918a-eddc34b42a0d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.589057 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.589036 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f714467d-5b58-497d-918a-eddc34b42a0d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.590828 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.590807 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f714467d-5b58-497d-918a-eddc34b42a0d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.590995 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.590977 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f714467d-5b58-497d-918a-eddc34b42a0d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.595720 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.595698 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv6p6\" (UniqueName: \"kubernetes.io/projected/f714467d-5b58-497d-918a-eddc34b42a0d-kube-api-access-hv6p6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h\" (UID: \"f714467d-5b58-497d-918a-eddc34b42a0d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.737740 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.737667 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:12.867480 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:12.867438 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h"] Jun 08 15:51:12.869739 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:51:12.869702 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf714467d_5b58_497d_918a_eddc34b42a0d.slice/crio-75ae6e9ce6f9abe1e97970ede682c3a6d37c5d1633f15c9453f824268e02af87 WatchSource:0}: Error finding container 75ae6e9ce6f9abe1e97970ede682c3a6d37c5d1633f15c9453f824268e02af87: Status 404 returned error can't find the container with id 75ae6e9ce6f9abe1e97970ede682c3a6d37c5d1633f15c9453f824268e02af87 Jun 08 15:51:13.690622 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:13.690583 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" event={"ID":"f714467d-5b58-497d-918a-eddc34b42a0d","Type":"ContainerStarted","Data":"75ae6e9ce6f9abe1e97970ede682c3a6d37c5d1633f15c9453f824268e02af87"} Jun 08 15:51:16.426311 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:16.426278 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:51:16.706534 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:16.706431 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" event={"ID":"f714467d-5b58-497d-918a-eddc34b42a0d","Type":"ContainerStarted","Data":"66cc32da9f4e691d5b12c8bf64b6febe81180fdf1da209336d0525aa8a1389f5"} Jun 08 15:51:21.614217 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.614184 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg"] Jun 08 15:51:21.616608 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.616590 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.618621 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.618594 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 08 15:51:21.627828 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.627804 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg"] Jun 08 15:51:21.669861 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.669831 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49bf1765-0085-4ab0-826f-e3754060d339-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.669994 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.669925 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49bf1765-0085-4ab0-826f-e3754060d339-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.669994 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.669968 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49bf1765-0085-4ab0-826f-e3754060d339-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.669994 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.669990 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9r4s\" (UniqueName: \"kubernetes.io/projected/49bf1765-0085-4ab0-826f-e3754060d339-kube-api-access-p9r4s\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.670179 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.670054 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/49bf1765-0085-4ab0-826f-e3754060d339-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.670179 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.670085 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49bf1765-0085-4ab0-826f-e3754060d339-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.700403 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.700375 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:51:21.770664 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.770636 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/49bf1765-0085-4ab0-826f-e3754060d339-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.770833 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.770675 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49bf1765-0085-4ab0-826f-e3754060d339-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.770833 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.770713 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49bf1765-0085-4ab0-826f-e3754060d339-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.770833 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.770763 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49bf1765-0085-4ab0-826f-e3754060d339-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.770833 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.770796 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49bf1765-0085-4ab0-826f-e3754060d339-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.770833 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.770826 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9r4s\" (UniqueName: \"kubernetes.io/projected/49bf1765-0085-4ab0-826f-e3754060d339-kube-api-access-p9r4s\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.771140 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.771120 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/49bf1765-0085-4ab0-826f-e3754060d339-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.771197 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.771162 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49bf1765-0085-4ab0-826f-e3754060d339-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.771323 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.771287 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49bf1765-0085-4ab0-826f-e3754060d339-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.773099 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.773078 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49bf1765-0085-4ab0-826f-e3754060d339-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.773195 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.773179 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49bf1765-0085-4ab0-826f-e3754060d339-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.782670 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.782646 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9r4s\" (UniqueName: \"kubernetes.io/projected/49bf1765-0085-4ab0-826f-e3754060d339-kube-api-access-p9r4s\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-sltzg\" (UID: \"49bf1765-0085-4ab0-826f-e3754060d339\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:21.926656 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:21.926617 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:22.058848 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:22.058818 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg"] Jun 08 15:51:22.062320 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:51:22.062297 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49bf1765_0085_4ab0_826f_e3754060d339.slice/crio-30f6e1d23249511ec5a5e381947582412d5c6d235ea399df76f2f93e37971886 WatchSource:0}: Error finding container 30f6e1d23249511ec5a5e381947582412d5c6d235ea399df76f2f93e37971886: Status 404 returned error can't find the container with id 30f6e1d23249511ec5a5e381947582412d5c6d235ea399df76f2f93e37971886 Jun 08 15:51:22.736291 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:22.736251 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" event={"ID":"49bf1765-0085-4ab0-826f-e3754060d339","Type":"ContainerStarted","Data":"d8613016f95997008f9e2f4204334e11fa0a2c0aa0d6261bd8cc469561855f20"} Jun 08 15:51:22.736708 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:22.736306 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" event={"ID":"49bf1765-0085-4ab0-826f-e3754060d339","Type":"ContainerStarted","Data":"30f6e1d23249511ec5a5e381947582412d5c6d235ea399df76f2f93e37971886"} Jun 08 15:51:23.743273 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:23.743239 2557 generic.go:356] "Generic (PLEG): container finished" podID="49bf1765-0085-4ab0-826f-e3754060d339" containerID="d8613016f95997008f9e2f4204334e11fa0a2c0aa0d6261bd8cc469561855f20" exitCode=0 Jun 08 15:51:23.743787 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:23.743290 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" event={"ID":"49bf1765-0085-4ab0-826f-e3754060d339","Type":"ContainerDied","Data":"d8613016f95997008f9e2f4204334e11fa0a2c0aa0d6261bd8cc469561855f20"} Jun 08 15:51:24.751175 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:24.751138 2557 generic.go:356] "Generic (PLEG): container finished" podID="f714467d-5b58-497d-918a-eddc34b42a0d" containerID="66cc32da9f4e691d5b12c8bf64b6febe81180fdf1da209336d0525aa8a1389f5" exitCode=0 Jun 08 15:51:24.751655 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:24.751215 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" event={"ID":"f714467d-5b58-497d-918a-eddc34b42a0d","Type":"ContainerDied","Data":"66cc32da9f4e691d5b12c8bf64b6febe81180fdf1da209336d0525aa8a1389f5"} Jun 08 15:51:25.757810 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:25.757769 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" event={"ID":"f714467d-5b58-497d-918a-eddc34b42a0d","Type":"ContainerStarted","Data":"2f38ec0efa10ff067ad5923d6fdf9cfa5367867ef45d5c04c699caddb784bf2a"} Jun 08 15:51:25.759494 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:25.759456 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" event={"ID":"49bf1765-0085-4ab0-826f-e3754060d339","Type":"ContainerStarted","Data":"bd15f7dd2e793f64c26167deee602393bc7eb445764ce9da2c086423da877405"} Jun 08 15:51:25.775332 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:25.775291 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" podStartSLOduration=2.724174621 podStartE2EDuration="13.775275413s" totalImagesPullingTime="3.861103725s" totalInitContainerRuntime="7.189997067s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:51:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:51:25.774237705 +0000 UTC m=+1075.863368783" watchObservedRunningTime="2026-06-08 15:51:25.775275413 +0000 UTC m=+1075.864406464" Jun 08 15:51:25.792011 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:25.791963 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" podStartSLOduration=2.414516036 podStartE2EDuration="4.791948331s" totalImagesPullingTime="1.080305005s" totalInitContainerRuntime="1.29712729s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:51:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:51:25.790214328 +0000 UTC m=+1075.879345388" watchObservedRunningTime="2026-06-08 15:51:25.791948331 +0000 UTC m=+1075.881079380" Jun 08 15:51:31.926823 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:31.926788 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:31.927265 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:31.926843 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:31.939573 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:31.939551 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:51:32.737899 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:32.737869 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:32.738057 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:32.737934 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:32.750324 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:32.750296 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:32.807817 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:32.807787 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h" Jun 08 15:51:32.808640 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:51:32.808620 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-sltzg" Jun 08 15:52:06.052258 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:52:06.052219 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:52:06.154501 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:52:06.154429 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:52:07.098834 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:52:07.098799 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:52:07.198997 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:52:07.198969 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:52:21.008946 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:52:21.008903 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:53:23.795384 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:23.795319 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:53:24.803739 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:24.803699 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:53:30.368208 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:30.368180 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 15:53:30.369418 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:30.369395 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 15:53:30.375790 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:30.375771 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 15:53:30.377084 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:30.377065 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 15:53:35.200092 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:35.200053 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:53:35.290819 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:35.290789 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6454c6fdb-gpk4j"] Jun 08 15:53:35.291050 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:35.291014 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6454c6fdb-gpk4j" podUID="5bae33f1-3a8c-4623-b170-fcc5da1031fb" containerName="manager" containerID="cri-o://d2fe390687475fd5d0af29e88df435bac74e0d767bfe8816879af16670d022d3" gracePeriod=30 Jun 08 15:53:35.537344 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:35.537321 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6454c6fdb-gpk4j" Jun 08 15:53:35.644250 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:35.644222 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2t5sz\" (UniqueName: \"kubernetes.io/projected/5bae33f1-3a8c-4623-b170-fcc5da1031fb-kube-api-access-2t5sz\") pod \"5bae33f1-3a8c-4623-b170-fcc5da1031fb\" (UID: \"5bae33f1-3a8c-4623-b170-fcc5da1031fb\") " Jun 08 15:53:35.644415 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:35.644319 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5bae33f1-3a8c-4623-b170-fcc5da1031fb-webhook-cert\") pod \"5bae33f1-3a8c-4623-b170-fcc5da1031fb\" (UID: \"5bae33f1-3a8c-4623-b170-fcc5da1031fb\") " Jun 08 15:53:35.646268 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:35.646239 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bae33f1-3a8c-4623-b170-fcc5da1031fb-kube-api-access-2t5sz" (OuterVolumeSpecName: "kube-api-access-2t5sz") pod "5bae33f1-3a8c-4623-b170-fcc5da1031fb" (UID: "5bae33f1-3a8c-4623-b170-fcc5da1031fb"). InnerVolumeSpecName "kube-api-access-2t5sz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 15:53:35.646268 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:35.646243 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bae33f1-3a8c-4623-b170-fcc5da1031fb-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5bae33f1-3a8c-4623-b170-fcc5da1031fb" (UID: "5bae33f1-3a8c-4623-b170-fcc5da1031fb"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 15:53:35.745751 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:35.745680 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2t5sz\" (UniqueName: \"kubernetes.io/projected/5bae33f1-3a8c-4623-b170-fcc5da1031fb-kube-api-access-2t5sz\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:53:35.745751 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:35.745705 2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5bae33f1-3a8c-4623-b170-fcc5da1031fb-webhook-cert\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 15:53:36.375581 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:36.375546 2557 generic.go:356] "Generic (PLEG): container finished" podID="5bae33f1-3a8c-4623-b170-fcc5da1031fb" containerID="d2fe390687475fd5d0af29e88df435bac74e0d767bfe8816879af16670d022d3" exitCode=0 Jun 08 15:53:36.375971 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:36.375589 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6454c6fdb-gpk4j" Jun 08 15:53:36.375971 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:36.375585 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6454c6fdb-gpk4j" event={"ID":"5bae33f1-3a8c-4623-b170-fcc5da1031fb","Type":"ContainerDied","Data":"d2fe390687475fd5d0af29e88df435bac74e0d767bfe8816879af16670d022d3"} Jun 08 15:53:36.375971 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:36.375684 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6454c6fdb-gpk4j" event={"ID":"5bae33f1-3a8c-4623-b170-fcc5da1031fb","Type":"ContainerDied","Data":"36a227bcbb0bda37cd4ec5710507623f66e34ee0049e69fb6aa3b3eaadd38541"} Jun 08 15:53:36.375971 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:36.375712 2557 scope.go:117] "RemoveContainer" containerID="d2fe390687475fd5d0af29e88df435bac74e0d767bfe8816879af16670d022d3" Jun 08 15:53:36.384713 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:36.384695 2557 scope.go:117] "RemoveContainer" containerID="d2fe390687475fd5d0af29e88df435bac74e0d767bfe8816879af16670d022d3" Jun 08 15:53:36.384964 ip-10-0-134-81 kubenswrapper[2557]: E0608 15:53:36.384944 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2fe390687475fd5d0af29e88df435bac74e0d767bfe8816879af16670d022d3\": container with ID starting with d2fe390687475fd5d0af29e88df435bac74e0d767bfe8816879af16670d022d3 not found: ID does not exist" containerID="d2fe390687475fd5d0af29e88df435bac74e0d767bfe8816879af16670d022d3" Jun 08 15:53:36.385025 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:36.384977 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2fe390687475fd5d0af29e88df435bac74e0d767bfe8816879af16670d022d3"} err="failed to get container status \"d2fe390687475fd5d0af29e88df435bac74e0d767bfe8816879af16670d022d3\": rpc error: code = NotFound desc = could not find container \"d2fe390687475fd5d0af29e88df435bac74e0d767bfe8816879af16670d022d3\": container with ID starting with d2fe390687475fd5d0af29e88df435bac74e0d767bfe8816879af16670d022d3 not found: ID does not exist" Jun 08 15:53:36.392825 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:36.392798 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6454c6fdb-gpk4j"] Jun 08 15:53:36.398342 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:36.398320 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6454c6fdb-gpk4j"] Jun 08 15:53:37.012994 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.012962 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:53:37.127483 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.127432 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6454c6fdb-bshxn"] Jun 08 15:53:37.127852 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.127836 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bae33f1-3a8c-4623-b170-fcc5da1031fb" containerName="manager" Jun 08 15:53:37.127896 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.127854 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bae33f1-3a8c-4623-b170-fcc5da1031fb" containerName="manager" Jun 08 15:53:37.127931 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.127923 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bae33f1-3a8c-4623-b170-fcc5da1031fb" containerName="manager" Jun 08 15:53:37.130879 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.130861 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6454c6fdb-bshxn" Jun 08 15:53:37.133473 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.133445 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 08 15:53:37.133543 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.133474 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7gjml\"" Jun 08 15:53:37.133595 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.133444 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 08 15:53:37.140021 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.140001 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6454c6fdb-bshxn"] Jun 08 15:53:37.157236 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.157207 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db18b9a0-a8e8-473a-91e5-0603ed7ca6e1-webhook-cert\") pod \"maas-controller-6454c6fdb-bshxn\" (UID: \"db18b9a0-a8e8-473a-91e5-0603ed7ca6e1\") " pod="opendatahub/maas-controller-6454c6fdb-bshxn" Jun 08 15:53:37.157347 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.157250 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd5hk\" (UniqueName: \"kubernetes.io/projected/db18b9a0-a8e8-473a-91e5-0603ed7ca6e1-kube-api-access-dd5hk\") pod \"maas-controller-6454c6fdb-bshxn\" (UID: \"db18b9a0-a8e8-473a-91e5-0603ed7ca6e1\") " pod="opendatahub/maas-controller-6454c6fdb-bshxn" Jun 08 15:53:37.257987 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.257955 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db18b9a0-a8e8-473a-91e5-0603ed7ca6e1-webhook-cert\") pod \"maas-controller-6454c6fdb-bshxn\" (UID: \"db18b9a0-a8e8-473a-91e5-0603ed7ca6e1\") " pod="opendatahub/maas-controller-6454c6fdb-bshxn" Jun 08 15:53:37.258144 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.257994 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd5hk\" (UniqueName: \"kubernetes.io/projected/db18b9a0-a8e8-473a-91e5-0603ed7ca6e1-kube-api-access-dd5hk\") pod \"maas-controller-6454c6fdb-bshxn\" (UID: \"db18b9a0-a8e8-473a-91e5-0603ed7ca6e1\") " pod="opendatahub/maas-controller-6454c6fdb-bshxn" Jun 08 15:53:37.260559 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.260537 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db18b9a0-a8e8-473a-91e5-0603ed7ca6e1-webhook-cert\") pod \"maas-controller-6454c6fdb-bshxn\" (UID: \"db18b9a0-a8e8-473a-91e5-0603ed7ca6e1\") " pod="opendatahub/maas-controller-6454c6fdb-bshxn" Jun 08 15:53:37.266119 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.266071 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd5hk\" (UniqueName: \"kubernetes.io/projected/db18b9a0-a8e8-473a-91e5-0603ed7ca6e1-kube-api-access-dd5hk\") pod \"maas-controller-6454c6fdb-bshxn\" (UID: \"db18b9a0-a8e8-473a-91e5-0603ed7ca6e1\") " pod="opendatahub/maas-controller-6454c6fdb-bshxn" Jun 08 15:53:37.441982 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.441948 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6454c6fdb-bshxn" Jun 08 15:53:37.774889 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:37.774703 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6454c6fdb-bshxn"] Jun 08 15:53:37.777609 ip-10-0-134-81 kubenswrapper[2557]: W0608 15:53:37.777579 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb18b9a0_a8e8_473a_91e5_0603ed7ca6e1.slice/crio-59d36b25f1f04212d3f2e7ea5545dcc50ab3ba083bde34a07266e2af49aca5a1 WatchSource:0}: Error finding container 59d36b25f1f04212d3f2e7ea5545dcc50ab3ba083bde34a07266e2af49aca5a1: Status 404 returned error can't find the container with id 59d36b25f1f04212d3f2e7ea5545dcc50ab3ba083bde34a07266e2af49aca5a1 Jun 08 15:53:38.338148 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:38.338112 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bae33f1-3a8c-4623-b170-fcc5da1031fb" path="/var/lib/kubelet/pods/5bae33f1-3a8c-4623-b170-fcc5da1031fb/volumes" Jun 08 15:53:38.386782 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:38.386749 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6454c6fdb-bshxn" event={"ID":"db18b9a0-a8e8-473a-91e5-0603ed7ca6e1","Type":"ContainerStarted","Data":"0a3d1f5002967bdc9a735e658087ed7810b0e2c8214a91a93122dbd8b0b3bce4"} Jun 08 15:53:38.386940 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:38.386798 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6454c6fdb-bshxn" event={"ID":"db18b9a0-a8e8-473a-91e5-0603ed7ca6e1","Type":"ContainerStarted","Data":"59d36b25f1f04212d3f2e7ea5545dcc50ab3ba083bde34a07266e2af49aca5a1"} Jun 08 15:53:38.386940 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:38.386892 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6454c6fdb-bshxn" Jun 08 15:53:38.402089 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:38.402046 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6454c6fdb-bshxn" podStartSLOduration=0.939858654 podStartE2EDuration="1.40203306s" totalImagesPullingTime="462.174406ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 15:53:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 15:53:38.400617319 +0000 UTC m=+1208.489748369" watchObservedRunningTime="2026-06-08 15:53:38.40203306 +0000 UTC m=+1208.491164109" Jun 08 15:53:49.397208 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:53:49.397175 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6454c6fdb-bshxn" Jun 08 15:54:19.127506 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:54:19.127410 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:54:19.221286 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:54:19.221245 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:54:19.321136 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:54:19.321095 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:54:19.889380 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:54:19.889336 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:54:19.989993 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:54:19.989956 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:54:27.604665 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:54:27.604631 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:54:33.219067 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:54:33.219031 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:54:33.279676 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:54:33.279650 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:00.913973 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:00.913937 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:05.335596 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:05.335559 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:05.437245 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:05.437215 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:05.538517 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:05.538485 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:06.026323 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:06.026283 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:06.126329 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:06.126291 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:29.304993 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:29.304944 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:34.970483 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:34.970415 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:35.022302 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:35.022272 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:35.121114 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:35.121084 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:35.220064 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:35.220018 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:35.332610 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:35.332527 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:35.428038 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:35.428005 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:37.582252 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:37.582219 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:51.830163 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:51.830130 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:51.944286 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:51.944246 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:52.044495 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:52.044448 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:57.825879 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:57.825839 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:55:57.925386 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:55:57.925351 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:56:05.944070 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:56:05.944032 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:56:06.043865 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:56:06.043826 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:56:06.143659 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:56:06.143623 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:56:06.243972 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:56:06.243889 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:56:22.644930 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:56:22.644893 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:56:22.744893 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:56:22.744847 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:56:22.845120 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:56:22.845077 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:56:28.576958 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:56:28.576924 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:56:28.632644 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:56:28.632610 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:56:51.672119 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:56:51.672084 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:56:51.775436 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:56:51.775404 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:56:57.073140 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:56:57.073104 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:56:57.133064 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:56:57.133031 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:56:57.195593 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:56:57.195558 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:56:57.295638 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:56:57.295602 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:56:57.403953 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:56:57.403926 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:57:03.300764 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:57:03.300732 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:57:03.365817 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:57:03.365772 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:58:30.398446 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:58:30.398418 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 15:58:30.400719 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:58:30.400698 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 15:58:30.406561 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:58:30.406543 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 15:58:30.408812 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:58:30.408797 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 15:58:41.258851 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:58:41.258820 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:58:59.206280 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:58:59.206251 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:58:59.613738 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:58:59.613636 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 15:58:59.715114 ip-10-0-134-81 kubenswrapper[2557]: I0608 15:58:59.715073 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:00:00.136566 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:00:00.136530 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9"] Jun 08 16:00:00.139680 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:00:00.139663 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9" Jun 08 16:00:00.150764 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:00:00.150737 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9"] Jun 08 16:00:00.248182 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:00:00.248152 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc5h9\" (UniqueName: \"kubernetes.io/projected/4791a598-f069-4043-88d4-ee99f8dd24c2-kube-api-access-lc5h9\") pod \"maas-api-key-cleanup-29682240-5j5h9\" (UID: \"4791a598-f069-4043-88d4-ee99f8dd24c2\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9" Jun 08 16:00:00.349394 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:00:00.349358 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc5h9\" (UniqueName: \"kubernetes.io/projected/4791a598-f069-4043-88d4-ee99f8dd24c2-kube-api-access-lc5h9\") pod \"maas-api-key-cleanup-29682240-5j5h9\" (UID: \"4791a598-f069-4043-88d4-ee99f8dd24c2\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9" Jun 08 16:00:00.357740 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:00:00.357718 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc5h9\" (UniqueName: \"kubernetes.io/projected/4791a598-f069-4043-88d4-ee99f8dd24c2-kube-api-access-lc5h9\") pod \"maas-api-key-cleanup-29682240-5j5h9\" (UID: \"4791a598-f069-4043-88d4-ee99f8dd24c2\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9" Jun 08 16:00:00.449884 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:00:00.449812 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9" Jun 08 16:00:00.579575 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:00:00.579484 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9"] Jun 08 16:00:00.585578 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:00:00.585556 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 16:00:01.194006 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:00:01.193969 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9" event={"ID":"4791a598-f069-4043-88d4-ee99f8dd24c2","Type":"ContainerStarted","Data":"3eed276e9da70f280a5dab0e64b6d921f8c8c667ff6b58720342e4be28af30d6"} Jun 08 16:00:08.229725 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:00:08.229685 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9" event={"ID":"4791a598-f069-4043-88d4-ee99f8dd24c2","Type":"ContainerStarted","Data":"f15dfdaa0ba5405d9af56cea89f985839b32f08f6fb31185a3f7398131207c65"} Jun 08 16:00:08.245142 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:00:08.245095 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9" podStartSLOduration=1.695130549 podStartE2EDuration="8.245079979s" totalImagesPullingTime="6.54994943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:00:08.244157779 +0000 UTC m=+1598.333288829" watchObservedRunningTime="2026-06-08 16:00:08.245079979 +0000 UTC m=+1598.334211028" Jun 08 16:01:00.010257 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:00.010219 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9"] Jun 08 16:01:00.010791 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:00.010450 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9" podUID="4791a598-f069-4043-88d4-ee99f8dd24c2" containerName="cleanup" containerID="cri-o://f15dfdaa0ba5405d9af56cea89f985839b32f08f6fb31185a3f7398131207c65" gracePeriod=30 Jun 08 16:01:30.187820 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:30.187797 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9" Jun 08 16:01:30.257963 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:30.257936 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lc5h9\" (UniqueName: \"kubernetes.io/projected/4791a598-f069-4043-88d4-ee99f8dd24c2-kube-api-access-lc5h9\") pod \"4791a598-f069-4043-88d4-ee99f8dd24c2\" (UID: \"4791a598-f069-4043-88d4-ee99f8dd24c2\") " Jun 08 16:01:30.260075 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:30.260042 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4791a598-f069-4043-88d4-ee99f8dd24c2-kube-api-access-lc5h9" (OuterVolumeSpecName: "kube-api-access-lc5h9") pod "4791a598-f069-4043-88d4-ee99f8dd24c2" (UID: "4791a598-f069-4043-88d4-ee99f8dd24c2"). InnerVolumeSpecName "kube-api-access-lc5h9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 16:01:30.359208 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:30.359176 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lc5h9\" (UniqueName: \"kubernetes.io/projected/4791a598-f069-4043-88d4-ee99f8dd24c2-kube-api-access-lc5h9\") on node \"ip-10-0-134-81.ec2.internal\" DevicePath \"\"" Jun 08 16:01:30.621617 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:30.621508 2557 generic.go:356] "Generic (PLEG): container finished" podID="4791a598-f069-4043-88d4-ee99f8dd24c2" containerID="f15dfdaa0ba5405d9af56cea89f985839b32f08f6fb31185a3f7398131207c65" exitCode=137 Jun 08 16:01:30.621617 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:30.621581 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9" Jun 08 16:01:30.621617 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:30.621590 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9" event={"ID":"4791a598-f069-4043-88d4-ee99f8dd24c2","Type":"ContainerDied","Data":"f15dfdaa0ba5405d9af56cea89f985839b32f08f6fb31185a3f7398131207c65"} Jun 08 16:01:30.621900 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:30.621647 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9" event={"ID":"4791a598-f069-4043-88d4-ee99f8dd24c2","Type":"ContainerDied","Data":"3eed276e9da70f280a5dab0e64b6d921f8c8c667ff6b58720342e4be28af30d6"} Jun 08 16:01:30.621900 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:30.621670 2557 scope.go:117] "RemoveContainer" containerID="f15dfdaa0ba5405d9af56cea89f985839b32f08f6fb31185a3f7398131207c65" Jun 08 16:01:30.629976 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:30.629957 2557 scope.go:117] "RemoveContainer" containerID="f15dfdaa0ba5405d9af56cea89f985839b32f08f6fb31185a3f7398131207c65" Jun 08 16:01:30.630250 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:01:30.630233 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f15dfdaa0ba5405d9af56cea89f985839b32f08f6fb31185a3f7398131207c65\": container with ID starting with f15dfdaa0ba5405d9af56cea89f985839b32f08f6fb31185a3f7398131207c65 not found: ID does not exist" containerID="f15dfdaa0ba5405d9af56cea89f985839b32f08f6fb31185a3f7398131207c65" Jun 08 16:01:30.630332 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:30.630259 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f15dfdaa0ba5405d9af56cea89f985839b32f08f6fb31185a3f7398131207c65"} err="failed to get container status \"f15dfdaa0ba5405d9af56cea89f985839b32f08f6fb31185a3f7398131207c65\": rpc error: code = NotFound desc = could not find container \"f15dfdaa0ba5405d9af56cea89f985839b32f08f6fb31185a3f7398131207c65\": container with ID starting with f15dfdaa0ba5405d9af56cea89f985839b32f08f6fb31185a3f7398131207c65 not found: ID does not exist" Jun 08 16:01:30.636624 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:30.636584 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9"] Jun 08 16:01:30.638294 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:30.638269 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29682240-5j5h9"] Jun 08 16:01:32.337996 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:32.337963 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4791a598-f069-4043-88d4-ee99f8dd24c2" path="/var/lib/kubelet/pods/4791a598-f069-4043-88d4-ee99f8dd24c2/volumes" Jun 08 16:01:57.929986 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:57.929952 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:01:58.018579 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:01:58.018543 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:02:54.003919 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:02:54.003882 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:03:30.428974 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:03:30.428942 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 16:03:30.432070 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:03:30.432050 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 16:03:30.436449 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:03:30.436428 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 16:03:30.439323 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:03:30.439308 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 16:04:36.102946 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:04:36.102908 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:04:37.191170 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:04:37.191138 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:07:49.174162 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:07:49.174128 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:08:05.520736 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:08:05.520703 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:08:30.458432 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:08:30.458405 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 16:08:30.465579 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:08:30.465553 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 16:08:30.469050 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:08:30.469030 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 16:08:30.473540 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:08:30.473523 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 16:09:56.414035 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:09:56.414002 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:13:16.686308 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:13:16.686275 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:13:30.492742 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:13:30.492703 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 16:13:30.498654 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:13:30.498631 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 16:13:30.501648 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:13:30.501630 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 16:13:30.506189 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:13:30.506175 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 16:14:02.643339 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:02.643297 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:08.015157 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:08.015100 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:32.271366 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:32.271330 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:45.313804 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:45.313752 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:46.928546 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:46.928519 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:47.031779 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.031735 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:47.129980 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.129945 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:47.205936 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.205856 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw"] Jun 08 16:14:47.207250 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.207218 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4791a598-f069-4043-88d4-ee99f8dd24c2" containerName="cleanup" Jun 08 16:14:47.207433 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.207417 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4791a598-f069-4043-88d4-ee99f8dd24c2" containerName="cleanup" Jun 08 16:14:47.207766 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.207750 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4791a598-f069-4043-88d4-ee99f8dd24c2" containerName="cleanup" Jun 08 16:14:47.215207 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.215185 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" Jun 08 16:14:47.218570 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.218538 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw"] Jun 08 16:14:47.220710 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.220683 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:47.256008 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.255978 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42923b6e-40db-4659-b2c6-98b41ea503cf-maas-api-tls\") pod \"maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw\" (UID: \"42923b6e-40db-4659-b2c6-98b41ea503cf\") " pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" Jun 08 16:14:47.256147 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.256123 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6mw4\" (UniqueName: \"kubernetes.io/projected/42923b6e-40db-4659-b2c6-98b41ea503cf-kube-api-access-g6mw4\") pod \"maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw\" (UID: \"42923b6e-40db-4659-b2c6-98b41ea503cf\") " pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" Jun 08 16:14:47.289183 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.289151 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:47.356643 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.356611 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6mw4\" (UniqueName: \"kubernetes.io/projected/42923b6e-40db-4659-b2c6-98b41ea503cf-kube-api-access-g6mw4\") pod \"maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw\" (UID: \"42923b6e-40db-4659-b2c6-98b41ea503cf\") " pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" Jun 08 16:14:47.356826 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.356653 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42923b6e-40db-4659-b2c6-98b41ea503cf-maas-api-tls\") pod \"maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw\" (UID: \"42923b6e-40db-4659-b2c6-98b41ea503cf\") " pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" Jun 08 16:14:47.359076 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.359053 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42923b6e-40db-4659-b2c6-98b41ea503cf-maas-api-tls\") pod \"maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw\" (UID: \"42923b6e-40db-4659-b2c6-98b41ea503cf\") " pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" Jun 08 16:14:47.367382 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.367357 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6mw4\" (UniqueName: \"kubernetes.io/projected/42923b6e-40db-4659-b2c6-98b41ea503cf-kube-api-access-g6mw4\") pod \"maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw\" (UID: \"42923b6e-40db-4659-b2c6-98b41ea503cf\") " pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" Jun 08 16:14:47.479112 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.478989 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:47.528657 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.528622 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" Jun 08 16:14:47.579652 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.579605 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:47.673973 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.673927 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw"] Jun 08 16:14:47.677211 ip-10-0-134-81 kubenswrapper[2557]: W0608 16:14:47.677180 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42923b6e_40db_4659_b2c6_98b41ea503cf.slice/crio-b0870883b28f42e63d90adee26db25f646c6b3ac6afcdf36d5ee1dbac73b25dc WatchSource:0}: Error finding container b0870883b28f42e63d90adee26db25f646c6b3ac6afcdf36d5ee1dbac73b25dc: Status 404 returned error can't find the container with id b0870883b28f42e63d90adee26db25f646c6b3ac6afcdf36d5ee1dbac73b25dc Jun 08 16:14:47.678536 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.678514 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 16:14:47.778768 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.778681 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:47.885664 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:47.885627 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:48.124667 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:48.124640 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:48.307905 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:48.307849 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:48.400978 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:48.400897 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" event={"ID":"42923b6e-40db-4659-b2c6-98b41ea503cf","Type":"ContainerStarted","Data":"2bb3c4b1b1a6b0e4db2bf28cc4179af746f2bc0a20103e469e777e5dab2d23e9"} Jun 08 16:14:48.400978 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:48.400946 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" event={"ID":"42923b6e-40db-4659-b2c6-98b41ea503cf","Type":"ContainerStarted","Data":"b0870883b28f42e63d90adee26db25f646c6b3ac6afcdf36d5ee1dbac73b25dc"} Jun 08 16:14:48.401155 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:48.401021 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" Jun 08 16:14:48.415821 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:48.415771 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" podStartSLOduration=0.944440616 podStartE2EDuration="1.415757328s" totalImagesPullingTime="471.316712ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:14:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:14:48.414848582 +0000 UTC m=+2478.503979632" watchObservedRunningTime="2026-06-08 16:14:48.415757328 +0000 UTC m=+2478.504888378" Jun 08 16:14:48.486364 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:48.486328 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:49.163448 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:49.163412 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:49.408099 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:49.408058 2557 generic.go:356] "Generic (PLEG): container finished" podID="42923b6e-40db-4659-b2c6-98b41ea503cf" containerID="2bb3c4b1b1a6b0e4db2bf28cc4179af746f2bc0a20103e469e777e5dab2d23e9" exitCode=1 Jun 08 16:14:49.408099 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:49.408104 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" event={"ID":"42923b6e-40db-4659-b2c6-98b41ea503cf","Type":"ContainerDied","Data":"2bb3c4b1b1a6b0e4db2bf28cc4179af746f2bc0a20103e469e777e5dab2d23e9"} Jun 08 16:14:49.408423 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:49.408404 2557 scope.go:117] "RemoveContainer" containerID="2bb3c4b1b1a6b0e4db2bf28cc4179af746f2bc0a20103e469e777e5dab2d23e9" Jun 08 16:14:49.866243 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:49.866216 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:50.414242 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:50.414213 2557 generic.go:356] "Generic (PLEG): container finished" podID="42923b6e-40db-4659-b2c6-98b41ea503cf" containerID="d6d0d69186a556886dbc871bcf38d2f6d41a0407b9d5df80eec590516c2aa8d8" exitCode=1 Jun 08 16:14:50.414689 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:50.414272 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" event={"ID":"42923b6e-40db-4659-b2c6-98b41ea503cf","Type":"ContainerDied","Data":"d6d0d69186a556886dbc871bcf38d2f6d41a0407b9d5df80eec590516c2aa8d8"} Jun 08 16:14:50.414689 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:50.414312 2557 scope.go:117] "RemoveContainer" containerID="2bb3c4b1b1a6b0e4db2bf28cc4179af746f2bc0a20103e469e777e5dab2d23e9" Jun 08 16:14:50.414689 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:50.414518 2557 scope.go:117] "RemoveContainer" containerID="d6d0d69186a556886dbc871bcf38d2f6d41a0407b9d5df80eec590516c2aa8d8" Jun 08 16:14:50.414838 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:14:50.414742 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw_redhat-ai-gateway-infra(42923b6e-40db-4659-b2c6-98b41ea503cf)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" podUID="42923b6e-40db-4659-b2c6-98b41ea503cf" Jun 08 16:14:50.528270 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:50.528237 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:50.696997 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:50.696967 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:50.852244 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:50.852218 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:51.421412 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:51.421381 2557 scope.go:117] "RemoveContainer" containerID="d6d0d69186a556886dbc871bcf38d2f6d41a0407b9d5df80eec590516c2aa8d8" Jun 08 16:14:51.421851 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:14:51.421615 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw_redhat-ai-gateway-infra(42923b6e-40db-4659-b2c6-98b41ea503cf)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" podUID="42923b6e-40db-4659-b2c6-98b41ea503cf" Jun 08 16:14:51.464890 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:51.464849 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:53.194484 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:53.194431 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:53.378641 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:53.378611 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:53.401515 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:53.401484 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" Jun 08 16:14:53.401930 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:53.401913 2557 scope.go:117] "RemoveContainer" containerID="d6d0d69186a556886dbc871bcf38d2f6d41a0407b9d5df80eec590516c2aa8d8" Jun 08 16:14:53.402165 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:14:53.402144 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw_redhat-ai-gateway-infra(42923b6e-40db-4659-b2c6-98b41ea503cf)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" podUID="42923b6e-40db-4659-b2c6-98b41ea503cf" Jun 08 16:14:53.479147 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:53.479067 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:54.912968 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:54.912939 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:57.528795 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:57.528766 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" Jun 08 16:14:57.529197 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:57.529093 2557 scope.go:117] "RemoveContainer" containerID="d6d0d69186a556886dbc871bcf38d2f6d41a0407b9d5df80eec590516c2aa8d8" Jun 08 16:14:57.529319 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:14:57.529299 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw_redhat-ai-gateway-infra(42923b6e-40db-4659-b2c6-98b41ea503cf)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" podUID="42923b6e-40db-4659-b2c6-98b41ea503cf" Jun 08 16:14:59.163012 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:59.162983 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:14:59.603619 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:59.603479 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h"] Jun 08 16:14:59.606038 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:59.606016 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" Jun 08 16:14:59.615844 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:59.615824 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h"] Jun 08 16:14:59.667059 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:59.667021 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42ba36f4-bc43-461c-8c9c-6f9d9bf0981b-maas-api-tls\") pod \"maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h\" (UID: \"42ba36f4-bc43-461c-8c9c-6f9d9bf0981b\") " pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" Jun 08 16:14:59.667219 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:59.667102 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76rcw\" (UniqueName: \"kubernetes.io/projected/42ba36f4-bc43-461c-8c9c-6f9d9bf0981b-kube-api-access-76rcw\") pod \"maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h\" (UID: \"42ba36f4-bc43-461c-8c9c-6f9d9bf0981b\") " pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" Jun 08 16:14:59.767622 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:59.767587 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76rcw\" (UniqueName: \"kubernetes.io/projected/42ba36f4-bc43-461c-8c9c-6f9d9bf0981b-kube-api-access-76rcw\") pod \"maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h\" (UID: \"42ba36f4-bc43-461c-8c9c-6f9d9bf0981b\") " pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" Jun 08 16:14:59.767795 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:59.767658 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42ba36f4-bc43-461c-8c9c-6f9d9bf0981b-maas-api-tls\") pod \"maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h\" (UID: \"42ba36f4-bc43-461c-8c9c-6f9d9bf0981b\") " pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" Jun 08 16:14:59.770065 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:59.770038 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42ba36f4-bc43-461c-8c9c-6f9d9bf0981b-maas-api-tls\") pod \"maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h\" (UID: \"42ba36f4-bc43-461c-8c9c-6f9d9bf0981b\") " pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" Jun 08 16:14:59.775912 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:59.775894 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76rcw\" (UniqueName: \"kubernetes.io/projected/42ba36f4-bc43-461c-8c9c-6f9d9bf0981b-kube-api-access-76rcw\") pod \"maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h\" (UID: \"42ba36f4-bc43-461c-8c9c-6f9d9bf0981b\") " pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" Jun 08 16:14:59.918332 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:14:59.918297 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" Jun 08 16:15:00.053779 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.053659 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h"] Jun 08 16:15:00.056216 ip-10-0-134-81 kubenswrapper[2557]: W0608 16:15:00.056189 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42ba36f4_bc43_461c_8c9c_6f9d9bf0981b.slice/crio-67e1444ba22e5bb5236074bab3bc816d2ba11b538728c82fb42e257a59bc44f2 WatchSource:0}: Error finding container 67e1444ba22e5bb5236074bab3bc816d2ba11b538728c82fb42e257a59bc44f2: Status 404 returned error can't find the container with id 67e1444ba22e5bb5236074bab3bc816d2ba11b538728c82fb42e257a59bc44f2 Jun 08 16:15:00.130676 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.130642 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-ba7922e6-29682255-x4s7d"] Jun 08 16:15:00.133939 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.133916 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-ba7922e6-29682255-x4s7d" Jun 08 16:15:00.134051 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.133918 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-eb8dc71e-29682255-25487"] Jun 08 16:15:00.136861 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.136840 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29682255-6tbpn"] Jun 08 16:15:00.136965 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.136949 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-eb8dc71e-29682255-25487" Jun 08 16:15:00.139014 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.138997 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682255-6tbpn" Jun 08 16:15:00.143508 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.143488 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-ba7922e6-29682255-x4s7d"] Jun 08 16:15:00.145956 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.145904 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-eb8dc71e-29682255-25487"] Jun 08 16:15:00.148323 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.148302 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29682255-6tbpn"] Jun 08 16:15:00.272599 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.272518 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vx24\" (UniqueName: \"kubernetes.io/projected/89cbfa9d-90e3-4874-bf5d-1caece959818-kube-api-access-6vx24\") pod \"maas-api-key-cleanup-29682255-6tbpn\" (UID: \"89cbfa9d-90e3-4874-bf5d-1caece959818\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682255-6tbpn" Jun 08 16:15:00.272599 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.272569 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hgqk\" (UniqueName: \"kubernetes.io/projected/a47fe71d-2fe1-427b-b654-f92247700e17-kube-api-access-2hgqk\") pod \"maas-api-key-cleanup-e2e-ait-eb8dc71e-29682255-25487\" (UID: \"a47fe71d-2fe1-427b-b654-f92247700e17\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-eb8dc71e-29682255-25487" Jun 08 16:15:00.272981 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.272652 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc79j\" (UniqueName: \"kubernetes.io/projected/5e496f43-9690-4a08-8775-0f4baa7710ee-kube-api-access-lc79j\") pod \"maas-api-key-cleanup-e2e-ait-ba7922e6-29682255-x4s7d\" (UID: \"5e496f43-9690-4a08-8775-0f4baa7710ee\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-ba7922e6-29682255-x4s7d" Jun 08 16:15:00.300436 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.300407 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:00.374031 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.374006 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hgqk\" (UniqueName: \"kubernetes.io/projected/a47fe71d-2fe1-427b-b654-f92247700e17-kube-api-access-2hgqk\") pod \"maas-api-key-cleanup-e2e-ait-eb8dc71e-29682255-25487\" (UID: \"a47fe71d-2fe1-427b-b654-f92247700e17\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-eb8dc71e-29682255-25487" Jun 08 16:15:00.374177 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.374051 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc79j\" (UniqueName: \"kubernetes.io/projected/5e496f43-9690-4a08-8775-0f4baa7710ee-kube-api-access-lc79j\") pod \"maas-api-key-cleanup-e2e-ait-ba7922e6-29682255-x4s7d\" (UID: \"5e496f43-9690-4a08-8775-0f4baa7710ee\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-ba7922e6-29682255-x4s7d" Jun 08 16:15:00.374177 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.374105 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vx24\" (UniqueName: \"kubernetes.io/projected/89cbfa9d-90e3-4874-bf5d-1caece959818-kube-api-access-6vx24\") pod \"maas-api-key-cleanup-29682255-6tbpn\" (UID: \"89cbfa9d-90e3-4874-bf5d-1caece959818\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682255-6tbpn" Jun 08 16:15:00.382226 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.382198 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hgqk\" (UniqueName: \"kubernetes.io/projected/a47fe71d-2fe1-427b-b654-f92247700e17-kube-api-access-2hgqk\") pod \"maas-api-key-cleanup-e2e-ait-eb8dc71e-29682255-25487\" (UID: \"a47fe71d-2fe1-427b-b654-f92247700e17\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-eb8dc71e-29682255-25487" Jun 08 16:15:00.382226 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.382220 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vx24\" (UniqueName: \"kubernetes.io/projected/89cbfa9d-90e3-4874-bf5d-1caece959818-kube-api-access-6vx24\") pod \"maas-api-key-cleanup-29682255-6tbpn\" (UID: \"89cbfa9d-90e3-4874-bf5d-1caece959818\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682255-6tbpn" Jun 08 16:15:00.382359 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.382252 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc79j\" (UniqueName: \"kubernetes.io/projected/5e496f43-9690-4a08-8775-0f4baa7710ee-kube-api-access-lc79j\") pod \"maas-api-key-cleanup-e2e-ait-ba7922e6-29682255-x4s7d\" (UID: \"5e496f43-9690-4a08-8775-0f4baa7710ee\") " pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-ba7922e6-29682255-x4s7d" Jun 08 16:15:00.396242 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.396220 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:00.448934 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.448912 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-ba7922e6-29682255-x4s7d" Jun 08 16:15:00.458026 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.458005 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-eb8dc71e-29682255-25487" Jun 08 16:15:00.466607 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.466576 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682255-6tbpn" Jun 08 16:15:00.469239 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.469216 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" event={"ID":"42ba36f4-bc43-461c-8c9c-6f9d9bf0981b","Type":"ContainerStarted","Data":"80d4fc3ff56708197244e53cb40e10a59e396eec2fa9df8b407cc589f3432da7"} Jun 08 16:15:00.469329 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.469269 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" event={"ID":"42ba36f4-bc43-461c-8c9c-6f9d9bf0981b","Type":"ContainerStarted","Data":"67e1444ba22e5bb5236074bab3bc816d2ba11b538728c82fb42e257a59bc44f2"} Jun 08 16:15:00.469329 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.469314 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" Jun 08 16:15:00.609910 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.609762 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" podStartSLOduration=1.267695903 podStartE2EDuration="1.609735584s" totalImagesPullingTime="342.039681ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:14:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:15:00.486038426 +0000 UTC m=+2490.575169489" watchObservedRunningTime="2026-06-08 16:15:00.609735584 +0000 UTC m=+2490.698866634" Jun 08 16:15:00.611314 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.611289 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-ba7922e6-29682255-x4s7d"] Jun 08 16:15:00.614556 ip-10-0-134-81 kubenswrapper[2557]: W0608 16:15:00.614517 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e496f43_9690_4a08_8775_0f4baa7710ee.slice/crio-31cd60a0fb86dc6c224348ddab6e470929bd89300107f29d0baafbc2af743e85 WatchSource:0}: Error finding container 31cd60a0fb86dc6c224348ddab6e470929bd89300107f29d0baafbc2af743e85: Status 404 returned error can't find the container with id 31cd60a0fb86dc6c224348ddab6e470929bd89300107f29d0baafbc2af743e85 Jun 08 16:15:00.667248 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.667220 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29682255-6tbpn"] Jun 08 16:15:00.669869 ip-10-0-134-81 kubenswrapper[2557]: W0608 16:15:00.669844 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89cbfa9d_90e3_4874_bf5d_1caece959818.slice/crio-81f0fc2ac1fab2960c974e74fb3c8c43f4f5114d5fec5f09c28e82eca24e834a WatchSource:0}: Error finding container 81f0fc2ac1fab2960c974e74fb3c8c43f4f5114d5fec5f09c28e82eca24e834a: Status 404 returned error can't find the container with id 81f0fc2ac1fab2960c974e74fb3c8c43f4f5114d5fec5f09c28e82eca24e834a Jun 08 16:15:00.843716 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:00.843639 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-eb8dc71e-29682255-25487"] Jun 08 16:15:00.849665 ip-10-0-134-81 kubenswrapper[2557]: W0608 16:15:00.849624 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda47fe71d_2fe1_427b_b654_f92247700e17.slice/crio-74195fcb8439cfca10653f29f91047bee3503f7cdb603c141399c4809673b452 WatchSource:0}: Error finding container 74195fcb8439cfca10653f29f91047bee3503f7cdb603c141399c4809673b452: Status 404 returned error can't find the container with id 74195fcb8439cfca10653f29f91047bee3503f7cdb603c141399c4809673b452 Jun 08 16:15:01.264188 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:01.264163 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:01.476094 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:01.476003 2557 generic.go:356] "Generic (PLEG): container finished" podID="42ba36f4-bc43-461c-8c9c-6f9d9bf0981b" containerID="80d4fc3ff56708197244e53cb40e10a59e396eec2fa9df8b407cc589f3432da7" exitCode=1 Jun 08 16:15:01.476500 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:01.476093 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" event={"ID":"42ba36f4-bc43-461c-8c9c-6f9d9bf0981b","Type":"ContainerDied","Data":"80d4fc3ff56708197244e53cb40e10a59e396eec2fa9df8b407cc589f3432da7"} Jun 08 16:15:01.476500 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:01.476289 2557 scope.go:117] "RemoveContainer" containerID="80d4fc3ff56708197244e53cb40e10a59e396eec2fa9df8b407cc589f3432da7" Jun 08 16:15:01.477778 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:01.477747 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-ba7922e6-29682255-x4s7d" event={"ID":"5e496f43-9690-4a08-8775-0f4baa7710ee","Type":"ContainerStarted","Data":"623312332b39392b54d57c5fc3d7996be76c796727e7e1ab545cd50b2152f7c4"} Jun 08 16:15:01.477903 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:01.477793 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-ba7922e6-29682255-x4s7d" event={"ID":"5e496f43-9690-4a08-8775-0f4baa7710ee","Type":"ContainerStarted","Data":"31cd60a0fb86dc6c224348ddab6e470929bd89300107f29d0baafbc2af743e85"} Jun 08 16:15:01.479268 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:01.479245 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-eb8dc71e-29682255-25487" event={"ID":"a47fe71d-2fe1-427b-b654-f92247700e17","Type":"ContainerStarted","Data":"436d014f28166c21d1c245452818d951b2c75c4092e4e031051217a3faa629b4"} Jun 08 16:15:01.479363 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:01.479289 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-eb8dc71e-29682255-25487" event={"ID":"a47fe71d-2fe1-427b-b654-f92247700e17","Type":"ContainerStarted","Data":"74195fcb8439cfca10653f29f91047bee3503f7cdb603c141399c4809673b452"} Jun 08 16:15:01.480682 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:01.480660 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682255-6tbpn" event={"ID":"89cbfa9d-90e3-4874-bf5d-1caece959818","Type":"ContainerStarted","Data":"d6e090e2c48773eb1d43f11b95909addbd946acb4f4fe5e9c22bd2d82550a640"} Jun 08 16:15:01.480805 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:01.480697 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682255-6tbpn" event={"ID":"89cbfa9d-90e3-4874-bf5d-1caece959818","Type":"ContainerStarted","Data":"81f0fc2ac1fab2960c974e74fb3c8c43f4f5114d5fec5f09c28e82eca24e834a"} Jun 08 16:15:01.507639 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:01.507451 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682255-6tbpn" podStartSLOduration=1.5074350239999998 podStartE2EDuration="1.507435024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:15:01.507204107 +0000 UTC m=+2491.596335159" watchObservedRunningTime="2026-06-08 16:15:01.507435024 +0000 UTC m=+2491.596566075" Jun 08 16:15:01.521367 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:01.521309 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-ba7922e6-29682255-x4s7d" podStartSLOduration=1.521293831 podStartE2EDuration="1.521293831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:15:01.52042073 +0000 UTC m=+2491.609551790" watchObservedRunningTime="2026-06-08 16:15:01.521293831 +0000 UTC m=+2491.610424883" Jun 08 16:15:01.533592 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:01.533544 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-eb8dc71e-29682255-25487" podStartSLOduration=1.5335323779999999 podStartE2EDuration="1.533532378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:15:01.532897755 +0000 UTC m=+2491.622028805" watchObservedRunningTime="2026-06-08 16:15:01.533532378 +0000 UTC m=+2491.622663450" Jun 08 16:15:01.644293 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:01.644260 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:02.480473 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:02.480403 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:02.487163 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:02.487133 2557 generic.go:356] "Generic (PLEG): container finished" podID="42ba36f4-bc43-461c-8c9c-6f9d9bf0981b" containerID="53aa7131a7b22a4aaaf284ec0f515689f2e249d8cf3786894575e93ac169b1e6" exitCode=1 Jun 08 16:15:02.487306 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:02.487243 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" event={"ID":"42ba36f4-bc43-461c-8c9c-6f9d9bf0981b","Type":"ContainerDied","Data":"53aa7131a7b22a4aaaf284ec0f515689f2e249d8cf3786894575e93ac169b1e6"} Jun 08 16:15:02.487374 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:02.487312 2557 scope.go:117] "RemoveContainer" containerID="80d4fc3ff56708197244e53cb40e10a59e396eec2fa9df8b407cc589f3432da7" Jun 08 16:15:02.487429 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:02.487385 2557 scope.go:117] "RemoveContainer" containerID="53aa7131a7b22a4aaaf284ec0f515689f2e249d8cf3786894575e93ac169b1e6" Jun 08 16:15:02.487651 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:02.487623 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h_redhat-ai-gateway-infra(42ba36f4-bc43-461c-8c9c-6f9d9bf0981b)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" podUID="42ba36f4-bc43-461c-8c9c-6f9d9bf0981b" Jun 08 16:15:02.774958 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:02.774922 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:02.828206 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:02.828173 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:03.437659 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:03.437628 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:03.494100 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:03.494075 2557 scope.go:117] "RemoveContainer" containerID="53aa7131a7b22a4aaaf284ec0f515689f2e249d8cf3786894575e93ac169b1e6" Jun 08 16:15:03.494487 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:03.494245 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h_redhat-ai-gateway-infra(42ba36f4-bc43-461c-8c9c-6f9d9bf0981b)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" podUID="42ba36f4-bc43-461c-8c9c-6f9d9bf0981b" Jun 08 16:15:03.607084 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:03.607054 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:04.405849 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:04.405815 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:05.108300 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:05.108261 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:05.470006 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:05.469960 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" Jun 08 16:15:05.470387 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:05.470373 2557 scope.go:117] "RemoveContainer" containerID="53aa7131a7b22a4aaaf284ec0f515689f2e249d8cf3786894575e93ac169b1e6" Jun 08 16:15:05.470594 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:05.470575 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h_redhat-ai-gateway-infra(42ba36f4-bc43-461c-8c9c-6f9d9bf0981b)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" podUID="42ba36f4-bc43-461c-8c9c-6f9d9bf0981b" Jun 08 16:15:05.677690 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:05.677661 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:06.859426 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:06.859395 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:06.959263 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:06.959194 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:09.918895 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:09.918863 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" Jun 08 16:15:09.919274 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:09.919240 2557 scope.go:117] "RemoveContainer" containerID="53aa7131a7b22a4aaaf284ec0f515689f2e249d8cf3786894575e93ac169b1e6" Jun 08 16:15:09.919433 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:09.919410 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h_redhat-ai-gateway-infra(42ba36f4-bc43-461c-8c9c-6f9d9bf0981b)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" podUID="42ba36f4-bc43-461c-8c9c-6f9d9bf0981b" Jun 08 16:15:10.442739 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:10.442710 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:12.332552 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:12.332521 2557 scope.go:117] "RemoveContainer" containerID="d6d0d69186a556886dbc871bcf38d2f6d41a0407b9d5df80eec590516c2aa8d8" Jun 08 16:15:12.719977 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:12.719944 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:12.820743 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:12.820723 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:13.544926 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:13.544896 2557 generic.go:356] "Generic (PLEG): container finished" podID="42923b6e-40db-4659-b2c6-98b41ea503cf" containerID="e7b5e51fe3aac8dec906f42c8aa46dfdbfd2fa265d291cb5fcde70a9cc355784" exitCode=1 Jun 08 16:15:13.545359 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:13.544973 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" event={"ID":"42923b6e-40db-4659-b2c6-98b41ea503cf","Type":"ContainerDied","Data":"e7b5e51fe3aac8dec906f42c8aa46dfdbfd2fa265d291cb5fcde70a9cc355784"} Jun 08 16:15:13.545359 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:13.545051 2557 scope.go:117] "RemoveContainer" containerID="d6d0d69186a556886dbc871bcf38d2f6d41a0407b9d5df80eec590516c2aa8d8" Jun 08 16:15:13.545486 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:13.545402 2557 scope.go:117] "RemoveContainer" containerID="e7b5e51fe3aac8dec906f42c8aa46dfdbfd2fa265d291cb5fcde70a9cc355784" Jun 08 16:15:13.545648 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:13.545626 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw_redhat-ai-gateway-infra(42923b6e-40db-4659-b2c6-98b41ea503cf)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" podUID="42923b6e-40db-4659-b2c6-98b41ea503cf" Jun 08 16:15:17.529618 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:17.529584 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" Jun 08 16:15:17.530129 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:17.530036 2557 scope.go:117] "RemoveContainer" containerID="e7b5e51fe3aac8dec906f42c8aa46dfdbfd2fa265d291cb5fcde70a9cc355784" Jun 08 16:15:17.530258 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:17.530238 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw_redhat-ai-gateway-infra(42923b6e-40db-4659-b2c6-98b41ea503cf)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" podUID="42923b6e-40db-4659-b2c6-98b41ea503cf" Jun 08 16:15:18.401857 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:18.401823 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" Jun 08 16:15:18.402204 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:18.402187 2557 scope.go:117] "RemoveContainer" containerID="e7b5e51fe3aac8dec906f42c8aa46dfdbfd2fa265d291cb5fcde70a9cc355784" Jun 08 16:15:18.402416 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:18.402395 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw_redhat-ai-gateway-infra(42923b6e-40db-4659-b2c6-98b41ea503cf)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" podUID="42923b6e-40db-4659-b2c6-98b41ea503cf" Jun 08 16:15:20.335103 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:20.335076 2557 scope.go:117] "RemoveContainer" containerID="53aa7131a7b22a4aaaf284ec0f515689f2e249d8cf3786894575e93ac169b1e6" Jun 08 16:15:21.588276 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:21.588231 2557 generic.go:356] "Generic (PLEG): container finished" podID="42ba36f4-bc43-461c-8c9c-6f9d9bf0981b" containerID="2c3f652ed7152b78e552c1b669f10ba56ae84d721786808ca3f049a6068ccfc1" exitCode=1 Jun 08 16:15:21.588756 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:21.588307 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" event={"ID":"42ba36f4-bc43-461c-8c9c-6f9d9bf0981b","Type":"ContainerDied","Data":"2c3f652ed7152b78e552c1b669f10ba56ae84d721786808ca3f049a6068ccfc1"} Jun 08 16:15:21.588756 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:21.588420 2557 scope.go:117] "RemoveContainer" containerID="53aa7131a7b22a4aaaf284ec0f515689f2e249d8cf3786894575e93ac169b1e6" Jun 08 16:15:21.588756 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:21.588705 2557 scope.go:117] "RemoveContainer" containerID="2c3f652ed7152b78e552c1b669f10ba56ae84d721786808ca3f049a6068ccfc1" Jun 08 16:15:21.588956 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:21.588899 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h_redhat-ai-gateway-infra(42ba36f4-bc43-461c-8c9c-6f9d9bf0981b)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" podUID="42ba36f4-bc43-461c-8c9c-6f9d9bf0981b" Jun 08 16:15:23.978438 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:23.978410 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6454c6fdb-bshxn_db18b9a0-a8e8-473a-91e5-0603ed7ca6e1/manager/0.log" Jun 08 16:15:24.592383 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:24.592357 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6484fb6dbf-qfw6d_cffbdb15-0b61-4124-8f64-2a22b45e65a2/copy-manifests/0.log" Jun 08 16:15:24.603012 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:24.602982 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6484fb6dbf-qfw6d_cffbdb15-0b61-4124-8f64-2a22b45e65a2/manager/0.log" Jun 08 16:15:25.469441 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:25.469407 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" Jun 08 16:15:25.469825 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:25.469783 2557 scope.go:117] "RemoveContainer" containerID="2c3f652ed7152b78e552c1b669f10ba56ae84d721786808ca3f049a6068ccfc1" Jun 08 16:15:25.470005 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:25.469986 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h_redhat-ai-gateway-infra(42ba36f4-bc43-461c-8c9c-6f9d9bf0981b)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" podUID="42ba36f4-bc43-461c-8c9c-6f9d9bf0981b" Jun 08 16:15:26.268326 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:26.268299 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-5xz5l_3b7867cd-55e9-4b0b-b6d0-5309761631aa/kuadrant-console-plugin/0.log" Jun 08 16:15:27.312155 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:27.312118 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74547644b9-jwfsk_07c7d369-f33d-4f07-a50f-9d10c42d5002/kube-auth-proxy/0.log" Jun 08 16:15:27.653640 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:27.653601 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-89lhg_48b98f5c-6b3b-44ab-a57f-1d77493c665c/payload-pre-processing/0.log" Jun 08 16:15:27.768697 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:27.768663 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-m42t7_06d84685-b8db-4564-a09a-da6ae5cc1017/payload-processing/0.log" Jun 08 16:15:27.877479 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:27.877443 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-664d9c4559-shg98_ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50/router/0.log" Jun 08 16:15:28.578602 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:28.578574 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h_f714467d-5b58-497d-918a-eddc34b42a0d/storage-initializer/0.log" Jun 08 16:15:28.585320 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:28.585300 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477czs29h_f714467d-5b58-497d-918a-eddc34b42a0d/main/0.log" Jun 08 16:15:28.818061 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:28.818035 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-sltzg_49bf1765-0085-4ab0-826f-e3754060d339/storage-initializer/0.log" Jun 08 16:15:28.825414 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:28.825389 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-sltzg_49bf1765-0085-4ab0-826f-e3754060d339/main/0.log" Jun 08 16:15:29.919340 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:29.919311 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" Jun 08 16:15:29.919844 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:29.919755 2557 scope.go:117] "RemoveContainer" containerID="2c3f652ed7152b78e552c1b669f10ba56ae84d721786808ca3f049a6068ccfc1" Jun 08 16:15:29.919972 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:29.919951 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h_redhat-ai-gateway-infra(42ba36f4-bc43-461c-8c9c-6f9d9bf0981b)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" podUID="42ba36f4-bc43-461c-8c9c-6f9d9bf0981b" Jun 08 16:15:32.332791 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:32.332760 2557 scope.go:117] "RemoveContainer" containerID="e7b5e51fe3aac8dec906f42c8aa46dfdbfd2fa265d291cb5fcde70a9cc355784" Jun 08 16:15:32.333162 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:32.332943 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw_redhat-ai-gateway-infra(42923b6e-40db-4659-b2c6-98b41ea503cf)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" podUID="42923b6e-40db-4659-b2c6-98b41ea503cf" Jun 08 16:15:32.582389 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:32.582357 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:32.676911 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:32.676876 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" limit=1024 Jun 08 16:15:35.788015 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:35.787991 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-r2mpb_1e9d36fd-7367-43bf-8a04-3675efc9957d/global-pull-secret-syncer/0.log" Jun 08 16:15:35.831335 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:35.831312 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8gf2f_ff5279a7-eec2-4e1b-a38f-ebee10911dd1/konnectivity-agent/0.log" Jun 08 16:15:35.920194 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:35.920174 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-81.ec2.internal_0d691a0e751fe5c9ef931276cec8b51d/haproxy/0.log" Jun 08 16:15:40.522838 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:40.522807 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-5xz5l_3b7867cd-55e9-4b0b-b6d0-5309761631aa/kuadrant-console-plugin/0.log" Jun 08 16:15:41.333089 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:41.333006 2557 scope.go:117] "RemoveContainer" containerID="2c3f652ed7152b78e552c1b669f10ba56ae84d721786808ca3f049a6068ccfc1" Jun 08 16:15:42.693073 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:42.693038 2557 generic.go:356] "Generic (PLEG): container finished" podID="42ba36f4-bc43-461c-8c9c-6f9d9bf0981b" containerID="c1dca8e692ccba2bc70996cc58f58ddd058d298cb7d15abe70b3e36629028bd9" exitCode=1 Jun 08 16:15:42.693536 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:42.693089 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" event={"ID":"42ba36f4-bc43-461c-8c9c-6f9d9bf0981b","Type":"ContainerDied","Data":"c1dca8e692ccba2bc70996cc58f58ddd058d298cb7d15abe70b3e36629028bd9"} Jun 08 16:15:42.693536 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:42.693148 2557 scope.go:117] "RemoveContainer" containerID="2c3f652ed7152b78e552c1b669f10ba56ae84d721786808ca3f049a6068ccfc1" Jun 08 16:15:42.693536 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:42.693450 2557 scope.go:117] "RemoveContainer" containerID="c1dca8e692ccba2bc70996cc58f58ddd058d298cb7d15abe70b3e36629028bd9" Jun 08 16:15:42.693736 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:42.693715 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h_redhat-ai-gateway-infra(42ba36f4-bc43-461c-8c9c-6f9d9bf0981b)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" podUID="42ba36f4-bc43-461c-8c9c-6f9d9bf0981b" Jun 08 16:15:42.798624 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:42.798599 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-ldc69_18622679-3a6c-45f7-adb0-7a2f922968c4/monitoring-plugin/0.log" Jun 08 16:15:42.902991 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:42.902964 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nf7qm_7578954d-074f-4b27-be5c-aa7cf8b194e7/node-exporter/0.log" Jun 08 16:15:42.922037 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:42.922019 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nf7qm_7578954d-074f-4b27-be5c-aa7cf8b194e7/kube-rbac-proxy/0.log" Jun 08 16:15:42.943381 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:42.943361 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nf7qm_7578954d-074f-4b27-be5c-aa7cf8b194e7/init-textfile/0.log" Jun 08 16:15:43.031609 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.031590 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-fk89c_73e378de-88ad-43cc-8078-ce02081e33d9/kube-rbac-proxy-main/0.log" Jun 08 16:15:43.051917 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.051866 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-fk89c_73e378de-88ad-43cc-8078-ce02081e33d9/kube-rbac-proxy-self/0.log" Jun 08 16:15:43.072123 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.072105 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-fk89c_73e378de-88ad-43cc-8078-ce02081e33d9/openshift-state-metrics/0.log" Jun 08 16:15:43.114606 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.114582 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82738f33-4db5-49b0-8d50-7012c6b4e13f/prometheus/0.log" Jun 08 16:15:43.133731 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.133709 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82738f33-4db5-49b0-8d50-7012c6b4e13f/config-reloader/0.log" Jun 08 16:15:43.152598 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.152576 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82738f33-4db5-49b0-8d50-7012c6b4e13f/thanos-sidecar/0.log" Jun 08 16:15:43.172036 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.172007 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82738f33-4db5-49b0-8d50-7012c6b4e13f/kube-rbac-proxy-web/0.log" Jun 08 16:15:43.191279 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.191243 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82738f33-4db5-49b0-8d50-7012c6b4e13f/kube-rbac-proxy/0.log" Jun 08 16:15:43.216301 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.216276 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82738f33-4db5-49b0-8d50-7012c6b4e13f/kube-rbac-proxy-thanos/0.log" Jun 08 16:15:43.235439 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.235420 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82738f33-4db5-49b0-8d50-7012c6b4e13f/init-config-reloader/0.log" Jun 08 16:15:43.261904 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.261885 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-mqq5n_de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736/prometheus-operator/0.log" Jun 08 16:15:43.279758 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.279738 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-mqq5n_de4cb47a-7cb5-4c7b-bc29-f9ab69fcc736/kube-rbac-proxy/0.log" Jun 08 16:15:43.307741 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.307689 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5b6b8f594-hqtpt_9d9afdc4-b5f0-4954-b58e-78441058f608/prometheus-operator-admission-webhook/0.log" Jun 08 16:15:43.429586 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.429568 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57b555fc9f-79kwk_742c5a1b-bf10-4705-857b-46c516917bf9/thanos-query/0.log" Jun 08 16:15:43.447965 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.447944 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57b555fc9f-79kwk_742c5a1b-bf10-4705-857b-46c516917bf9/kube-rbac-proxy-web/0.log" Jun 08 16:15:43.472345 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.472325 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57b555fc9f-79kwk_742c5a1b-bf10-4705-857b-46c516917bf9/kube-rbac-proxy/0.log" Jun 08 16:15:43.490890 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.490873 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57b555fc9f-79kwk_742c5a1b-bf10-4705-857b-46c516917bf9/prom-label-proxy/0.log" Jun 08 16:15:43.515382 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.515366 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57b555fc9f-79kwk_742c5a1b-bf10-4705-857b-46c516917bf9/kube-rbac-proxy-rules/0.log" Jun 08 16:15:43.533688 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:43.533672 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57b555fc9f-79kwk_742c5a1b-bf10-4705-857b-46c516917bf9/kube-rbac-proxy-metrics/0.log" Jun 08 16:15:45.470166 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:45.470137 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" Jun 08 16:15:45.470562 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:45.470437 2557 scope.go:117] "RemoveContainer" containerID="c1dca8e692ccba2bc70996cc58f58ddd058d298cb7d15abe70b3e36629028bd9" Jun 08 16:15:45.470659 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:45.470640 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h_redhat-ai-gateway-infra(42ba36f4-bc43-461c-8c9c-6f9d9bf0981b)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" podUID="42ba36f4-bc43-461c-8c9c-6f9d9bf0981b" Jun 08 16:15:45.731825 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:45.731737 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/2.log" Jun 08 16:15:45.736530 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:45.736508 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hhh67_2c03b1b7-10b4-43a0-a952-bd191d47aff5/console-operator/3.log" Jun 08 16:15:47.332189 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:47.332159 2557 scope.go:117] "RemoveContainer" containerID="e7b5e51fe3aac8dec906f42c8aa46dfdbfd2fa265d291cb5fcde70a9cc355784" Jun 08 16:15:47.722495 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:47.722448 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" event={"ID":"42923b6e-40db-4659-b2c6-98b41ea503cf","Type":"ContainerStarted","Data":"b55d077863bd9923fba9fc3b792c3b16a01215ae9cf613cae69151702037bba6"} Jun 08 16:15:47.722714 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:47.722697 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" Jun 08 16:15:48.303509 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.303422 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t8nc2_b9492b29-8d64-43ca-9f70-250f32707cfd/dns/0.log" Jun 08 16:15:48.322772 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.322751 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t8nc2_b9492b29-8d64-43ca-9f70-250f32707cfd/kube-rbac-proxy/0.log" Jun 08 16:15:48.369548 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.369519 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq"] Jun 08 16:15:48.372076 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.372058 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.373950 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.373932 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pxtpw\"/\"kube-root-ca.crt\"" Jun 08 16:15:48.374512 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.374497 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pxtpw\"/\"openshift-service-ca.crt\"" Jun 08 16:15:48.374601 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.374512 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pxtpw\"/\"default-dockercfg-mgwbq\"" Jun 08 16:15:48.381671 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.381648 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq"] Jun 08 16:15:48.416021 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.416004 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l8rfj_9865e636-5527-48fe-b4a8-de43e6b507ec/dns-node-resolver/0.log" Jun 08 16:15:48.499316 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.499293 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/66f9db91-b0fb-47ce-adc2-a7690bb0dd59-proc\") pod \"perf-node-gather-daemonset-mdwvq\" (UID: \"66f9db91-b0fb-47ce-adc2-a7690bb0dd59\") " pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.499430 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.499331 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/66f9db91-b0fb-47ce-adc2-a7690bb0dd59-podres\") pod \"perf-node-gather-daemonset-mdwvq\" (UID: \"66f9db91-b0fb-47ce-adc2-a7690bb0dd59\") " pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.499430 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.499380 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66f9db91-b0fb-47ce-adc2-a7690bb0dd59-sys\") pod \"perf-node-gather-daemonset-mdwvq\" (UID: \"66f9db91-b0fb-47ce-adc2-a7690bb0dd59\") " pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.499430 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.499405 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srqsl\" (UniqueName: \"kubernetes.io/projected/66f9db91-b0fb-47ce-adc2-a7690bb0dd59-kube-api-access-srqsl\") pod \"perf-node-gather-daemonset-mdwvq\" (UID: \"66f9db91-b0fb-47ce-adc2-a7690bb0dd59\") " pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.499559 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.499509 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66f9db91-b0fb-47ce-adc2-a7690bb0dd59-lib-modules\") pod \"perf-node-gather-daemonset-mdwvq\" (UID: \"66f9db91-b0fb-47ce-adc2-a7690bb0dd59\") " pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.600033 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.599950 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srqsl\" (UniqueName: \"kubernetes.io/projected/66f9db91-b0fb-47ce-adc2-a7690bb0dd59-kube-api-access-srqsl\") pod \"perf-node-gather-daemonset-mdwvq\" (UID: \"66f9db91-b0fb-47ce-adc2-a7690bb0dd59\") " pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.600033 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.600014 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66f9db91-b0fb-47ce-adc2-a7690bb0dd59-lib-modules\") pod \"perf-node-gather-daemonset-mdwvq\" (UID: \"66f9db91-b0fb-47ce-adc2-a7690bb0dd59\") " pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.600235 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.600053 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/66f9db91-b0fb-47ce-adc2-a7690bb0dd59-proc\") pod \"perf-node-gather-daemonset-mdwvq\" (UID: \"66f9db91-b0fb-47ce-adc2-a7690bb0dd59\") " pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.600235 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.600092 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/66f9db91-b0fb-47ce-adc2-a7690bb0dd59-podres\") pod \"perf-node-gather-daemonset-mdwvq\" (UID: \"66f9db91-b0fb-47ce-adc2-a7690bb0dd59\") " pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.600235 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.600144 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66f9db91-b0fb-47ce-adc2-a7690bb0dd59-sys\") pod \"perf-node-gather-daemonset-mdwvq\" (UID: \"66f9db91-b0fb-47ce-adc2-a7690bb0dd59\") " pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.600235 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.600191 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/66f9db91-b0fb-47ce-adc2-a7690bb0dd59-proc\") pod \"perf-node-gather-daemonset-mdwvq\" (UID: \"66f9db91-b0fb-47ce-adc2-a7690bb0dd59\") " pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.600235 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.600223 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66f9db91-b0fb-47ce-adc2-a7690bb0dd59-sys\") pod \"perf-node-gather-daemonset-mdwvq\" (UID: \"66f9db91-b0fb-47ce-adc2-a7690bb0dd59\") " pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.600235 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.600231 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66f9db91-b0fb-47ce-adc2-a7690bb0dd59-lib-modules\") pod \"perf-node-gather-daemonset-mdwvq\" (UID: \"66f9db91-b0fb-47ce-adc2-a7690bb0dd59\") " pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.600441 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.600277 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/66f9db91-b0fb-47ce-adc2-a7690bb0dd59-podres\") pod \"perf-node-gather-daemonset-mdwvq\" (UID: \"66f9db91-b0fb-47ce-adc2-a7690bb0dd59\") " pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.606777 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.606753 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srqsl\" (UniqueName: \"kubernetes.io/projected/66f9db91-b0fb-47ce-adc2-a7690bb0dd59-kube-api-access-srqsl\") pod \"perf-node-gather-daemonset-mdwvq\" (UID: \"66f9db91-b0fb-47ce-adc2-a7690bb0dd59\") " pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.682946 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.682914 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:48.731939 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.731901 2557 generic.go:356] "Generic (PLEG): container finished" podID="42923b6e-40db-4659-b2c6-98b41ea503cf" containerID="b55d077863bd9923fba9fc3b792c3b16a01215ae9cf613cae69151702037bba6" exitCode=1 Jun 08 16:15:48.732122 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.731984 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" event={"ID":"42923b6e-40db-4659-b2c6-98b41ea503cf","Type":"ContainerDied","Data":"b55d077863bd9923fba9fc3b792c3b16a01215ae9cf613cae69151702037bba6"} Jun 08 16:15:48.732122 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.732048 2557 scope.go:117] "RemoveContainer" containerID="e7b5e51fe3aac8dec906f42c8aa46dfdbfd2fa265d291cb5fcde70a9cc355784" Jun 08 16:15:48.732246 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:48.732229 2557 scope.go:117] "RemoveContainer" containerID="b55d077863bd9923fba9fc3b792c3b16a01215ae9cf613cae69151702037bba6" Jun 08 16:15:48.732496 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:48.732456 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw_redhat-ai-gateway-infra(42923b6e-40db-4659-b2c6-98b41ea503cf)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" podUID="42923b6e-40db-4659-b2c6-98b41ea503cf" Jun 08 16:15:49.025997 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:49.025937 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq"] Jun 08 16:15:49.028411 ip-10-0-134-81 kubenswrapper[2557]: W0608 16:15:49.028376 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod66f9db91_b0fb_47ce_adc2_a7690bb0dd59.slice/crio-c9c62a51071f65ffd891ab6f09e3dbac3d9b1fec6f76a8295c2fc081dfac1c4a WatchSource:0}: Error finding container c9c62a51071f65ffd891ab6f09e3dbac3d9b1fec6f76a8295c2fc081dfac1c4a: Status 404 returned error can't find the container with id c9c62a51071f65ffd891ab6f09e3dbac3d9b1fec6f76a8295c2fc081dfac1c4a Jun 08 16:15:49.125723 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:49.125701 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5t7fb_d777d945-3cc7-4e04-b64b-f021f129aea7/node-ca/0.log" Jun 08 16:15:49.738810 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:49.738781 2557 scope.go:117] "RemoveContainer" containerID="b55d077863bd9923fba9fc3b792c3b16a01215ae9cf613cae69151702037bba6" Jun 08 16:15:49.739238 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:49.739002 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw_redhat-ai-gateway-infra(42923b6e-40db-4659-b2c6-98b41ea503cf)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" podUID="42923b6e-40db-4659-b2c6-98b41ea503cf" Jun 08 16:15:49.740033 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:49.740008 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" event={"ID":"66f9db91-b0fb-47ce-adc2-a7690bb0dd59","Type":"ContainerStarted","Data":"2c8101f5e71524c0f8a03ea77e6bfbcad598058522501fd4d2153f5e8540e491"} Jun 08 16:15:49.740151 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:49.740057 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:49.740151 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:49.740073 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" event={"ID":"66f9db91-b0fb-47ce-adc2-a7690bb0dd59","Type":"ContainerStarted","Data":"c9c62a51071f65ffd891ab6f09e3dbac3d9b1fec6f76a8295c2fc081dfac1c4a"} Jun 08 16:15:49.768213 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:49.768164 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" podStartSLOduration=1.768150818 podStartE2EDuration="1.768150818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 16:15:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:15:49.767049782 +0000 UTC m=+2539.856180832" watchObservedRunningTime="2026-06-08 16:15:49.768150818 +0000 UTC m=+2539.857281867" Jun 08 16:15:49.918739 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:49.918712 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" Jun 08 16:15:49.919026 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:49.919013 2557 scope.go:117] "RemoveContainer" containerID="c1dca8e692ccba2bc70996cc58f58ddd058d298cb7d15abe70b3e36629028bd9" Jun 08 16:15:49.919186 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:49.919169 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h_redhat-ai-gateway-infra(42ba36f4-bc43-461c-8c9c-6f9d9bf0981b)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" podUID="42ba36f4-bc43-461c-8c9c-6f9d9bf0981b" Jun 08 16:15:50.183672 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:50.183641 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74547644b9-jwfsk_07c7d369-f33d-4f07-a50f-9d10c42d5002/kube-auth-proxy/0.log" Jun 08 16:15:50.314228 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:50.314202 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-89lhg_48b98f5c-6b3b-44ab-a57f-1d77493c665c/payload-pre-processing/0.log" Jun 08 16:15:50.371608 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:50.371582 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-m42t7_06d84685-b8db-4564-a09a-da6ae5cc1017/payload-processing/0.log" Jun 08 16:15:50.391785 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:50.391746 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-664d9c4559-shg98_ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50/router/0.log" Jun 08 16:15:50.998169 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:50.998133 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mvlgf_ac6d6aa0-340a-4156-aa2d-60444b6f4fcb/serve-healthcheck-canary/0.log" Jun 08 16:15:51.582864 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:51.582834 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wgrqt_09ff7cba-a236-4931-97aa-0c52dc0e2ec3/kube-rbac-proxy/0.log" Jun 08 16:15:51.602034 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:51.602007 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wgrqt_09ff7cba-a236-4931-97aa-0c52dc0e2ec3/exporter/0.log" Jun 08 16:15:51.621599 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:51.621578 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wgrqt_09ff7cba-a236-4931-97aa-0c52dc0e2ec3/extractor/0.log" Jun 08 16:15:54.066914 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:54.066880 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6454c6fdb-bshxn_db18b9a0-a8e8-473a-91e5-0603ed7ca6e1/manager/0.log" Jun 08 16:15:54.402729 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:54.402706 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6484fb6dbf-qfw6d_cffbdb15-0b61-4124-8f64-2a22b45e65a2/manager/0.log" Jun 08 16:15:54.421022 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:54.421003 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6484fb6dbf-qfw6d_cffbdb15-0b61-4124-8f64-2a22b45e65a2/copy-manifests/0.log" Jun 08 16:15:55.755659 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:55.755635 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pxtpw/perf-node-gather-daemonset-mdwvq" Jun 08 16:15:55.859998 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:55.859972 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-876584476-p5p64_6309c02c-b717-4e1e-b61d-66f15d87e0ca/manager/0.log" Jun 08 16:15:57.529608 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:57.529519 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" Jun 08 16:15:57.529950 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:15:57.529876 2557 scope.go:117] "RemoveContainer" containerID="b55d077863bd9923fba9fc3b792c3b16a01215ae9cf613cae69151702037bba6" Jun 08 16:15:57.530066 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:15:57.530047 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw_redhat-ai-gateway-infra(42923b6e-40db-4659-b2c6-98b41ea503cf)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-eb8dc71e-649c55f44c-lcbkw" podUID="42923b6e-40db-4659-b2c6-98b41ea503cf" Jun 08 16:16:00.010726 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:00.010635 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-eb8dc71e-29682255-25487"] Jun 08 16:16:00.011104 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:00.010855 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-eb8dc71e-29682255-25487" podUID="a47fe71d-2fe1-427b-b654-f92247700e17" containerName="cleanup" containerID="cri-o://436d014f28166c21d1c245452818d951b2c75c4092e4e031051217a3faa629b4" gracePeriod=30 Jun 08 16:16:00.012965 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:00.012939 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-ba7922e6-29682255-x4s7d"] Jun 08 16:16:00.013158 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:00.013137 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-ba7922e6-29682255-x4s7d" podUID="5e496f43-9690-4a08-8775-0f4baa7710ee" containerName="cleanup" containerID="cri-o://623312332b39392b54d57c5fc3d7996be76c796727e7e1ab545cd50b2152f7c4" gracePeriod=30 Jun 08 16:16:00.014889 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:00.014867 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["redhat-ai-gateway-infra/maas-api-key-cleanup-29682255-6tbpn"] Jun 08 16:16:00.015056 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:00.015036 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="redhat-ai-gateway-infra/maas-api-key-cleanup-29682255-6tbpn" podUID="89cbfa9d-90e3-4874-bf5d-1caece959818" containerName="cleanup" containerID="cri-o://d6e090e2c48773eb1d43f11b95909addbd946acb4f4fe5e9c22bd2d82550a640" gracePeriod=30 Jun 08 16:16:00.335387 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:00.335300 2557 scope.go:117] "RemoveContainer" containerID="c1dca8e692ccba2bc70996cc58f58ddd058d298cb7d15abe70b3e36629028bd9" Jun 08 16:16:00.335565 ip-10-0-134-81 kubenswrapper[2557]: E0608 16:16:00.335526 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h_redhat-ai-gateway-infra(42ba36f4-bc43-461c-8c9c-6f9d9bf0981b)\"" pod="redhat-ai-gateway-infra/maas-api-e2e-ait-ba7922e6-d59d74474-n5z2h" podUID="42ba36f4-bc43-461c-8c9c-6f9d9bf0981b" Jun 08 16:16:01.010409 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:01.010377 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74547644b9-jwfsk_07c7d369-f33d-4f07-a50f-9d10c42d5002/kube-auth-proxy/0.log" Jun 08 16:16:01.132478 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:01.132431 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-89lhg_48b98f5c-6b3b-44ab-a57f-1d77493c665c/payload-pre-processing/0.log" Jun 08 16:16:01.186946 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:01.186913 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-m42t7_06d84685-b8db-4564-a09a-da6ae5cc1017/payload-processing/0.log" Jun 08 16:16:01.201175 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:01.201149 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-664d9c4559-shg98_ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50/router/0.log" Jun 08 16:16:02.452108 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:02.452078 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74547644b9-jwfsk_07c7d369-f33d-4f07-a50f-9d10c42d5002/kube-auth-proxy/0.log" Jun 08 16:16:02.569100 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:02.569071 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-89lhg_48b98f5c-6b3b-44ab-a57f-1d77493c665c/payload-pre-processing/0.log" Jun 08 16:16:02.617047 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:02.617016 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-m42t7_06d84685-b8db-4564-a09a-da6ae5cc1017/payload-processing/0.log" Jun 08 16:16:02.632708 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:02.632682 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-664d9c4559-shg98_ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50/router/0.log" Jun 08 16:16:03.093717 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:03.093690 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tpwvw_a7cad45c-9b89-43d6-bd57-7b3194b1d7c3/kube-multus-additional-cni-plugins/0.log" Jun 08 16:16:03.113669 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:03.113640 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tpwvw_a7cad45c-9b89-43d6-bd57-7b3194b1d7c3/egress-router-binary-copy/0.log" Jun 08 16:16:03.134485 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:03.134430 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tpwvw_a7cad45c-9b89-43d6-bd57-7b3194b1d7c3/cni-plugins/0.log" Jun 08 16:16:03.153228 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:03.153202 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tpwvw_a7cad45c-9b89-43d6-bd57-7b3194b1d7c3/bond-cni-plugin/0.log" Jun 08 16:16:03.172435 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:03.172413 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tpwvw_a7cad45c-9b89-43d6-bd57-7b3194b1d7c3/routeoverride-cni/0.log" Jun 08 16:16:03.191851 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:03.191832 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tpwvw_a7cad45c-9b89-43d6-bd57-7b3194b1d7c3/whereabouts-cni-bincopy/0.log" Jun 08 16:16:03.211683 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:03.211653 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tpwvw_a7cad45c-9b89-43d6-bd57-7b3194b1d7c3/whereabouts-cni/0.log" Jun 08 16:16:03.244380 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:03.244354 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qmt5n_6a8b4c5c-e2c8-4a49-87b2-e80996d54193/kube-multus/0.log" Jun 08 16:16:03.295976 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:03.295939 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hct6z_75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58/network-metrics-daemon/0.log" Jun 08 16:16:03.314651 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:03.314628 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hct6z_75dc79a6-6f73-4df4-a4df-a7b6f3eb5f58/kube-rbac-proxy/0.log" Jun 08 16:16:04.277891 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:04.277863 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74547644b9-jwfsk_07c7d369-f33d-4f07-a50f-9d10c42d5002/kube-auth-proxy/0.log" Jun 08 16:16:04.382954 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:04.382926 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-89lhg_48b98f5c-6b3b-44ab-a57f-1d77493c665c/payload-pre-processing/0.log" Jun 08 16:16:04.428039 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:04.428008 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-m42t7_06d84685-b8db-4564-a09a-da6ae5cc1017/payload-processing/0.log" Jun 08 16:16:04.439579 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:04.439558 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-664d9c4559-shg98_ad9b4f9b-fe56-4de3-8c0c-0dd5ce50ce50/router/0.log" Jun 08 16:16:04.725471 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:04.725419 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-controller/0.log" Jun 08 16:16:04.748435 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:04.748405 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/1.log" Jun 08 16:16:04.748594 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:04.748480 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovn-acl-logging/0.log" Jun 08 16:16:04.767887 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:04.767860 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/kube-rbac-proxy-node/0.log" Jun 08 16:16:04.788127 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:04.788101 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/kube-rbac-proxy-ovn-metrics/0.log" Jun 08 16:16:04.804378 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:04.804358 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/northd/0.log" Jun 08 16:16:04.824070 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:04.824051 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/nbdb/0.log" Jun 08 16:16:04.843808 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:04.843793 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/sbdb/0.log" Jun 08 16:16:04.886829 ip-10-0-134-81 kubenswrapper[2557]: I0608 16:16:04.886807 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tsb88_efff1108-d45f-4dd8-9cf8-24cbbd122b87/ovnkube-controller/0.log"