Jun 03 23:21:49.992906 ip-10-0-129-110 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jun 03 23:21:49.992918 ip-10-0-129-110 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jun 03 23:21:49.992926 ip-10-0-129-110 systemd[1]: kubelet.service: Failed with result 'resources'.
Jun 03 23:21:49.993073 ip-10-0-129-110 systemd[1]: Failed to start Kubernetes Kubelet.
Jun 03 23:22:01.458183 ip-10-0-129-110 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jun 03 23:22:01.458199 ip-10-0-129-110 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 15e9cc07104e4ce9b9c274e6c90fbcaa --
Jun 03 23:24:37.713634 ip-10-0-129-110 systemd[1]: Starting Kubernetes Kubelet...
Jun 03 23:24:38.116183 ip-10-0-129-110 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 03 23:24:38.116183 ip-10-0-129-110 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jun 03 23:24:38.116183 ip-10-0-129-110 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 03 23:24:38.116183 ip-10-0-129-110 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jun 03 23:24:38.116183 ip-10-0-129-110 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 03 23:24:38.117089 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.117006    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jun 03 23:24:38.123450 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123425    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 03 23:24:38.123450 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123443    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 03 23:24:38.123450 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123447    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 03 23:24:38.123450 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123451    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 03 23:24:38.123450 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123454    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 03 23:24:38.123450 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123457    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123461    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123464    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123467    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123469    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123473    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123476    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123479    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123481    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123484    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123486    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123489    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123492    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123494    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123497    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123500    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123503    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123509    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123512    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123515    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 03 23:24:38.123673 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123517    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123520    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123523    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123526    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123529    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123532    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123535    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123537    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123540    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123543    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123545    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123548    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123551    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123556    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123561    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123564    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123568    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123571    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123574    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 03 23:24:38.124158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123577    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123580    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123583    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123586    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123588    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123591    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123594    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123597    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123599    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123603    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123606    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123610    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123613    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123616    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123618    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123621    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123624    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123627    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123629    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123632    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jun 03 23:24:38.124641 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123635    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123637    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123640    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123642    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123645    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123647    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123650    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123653    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123656    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123659    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123662    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123664    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123667    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123669    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123672    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123675    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123677    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123680    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123683    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 03 23:24:38.125158 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123686    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 03 23:24:38.125639 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123689    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 03 23:24:38.125639 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123692    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 03 23:24:38.125639 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123695    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 03 23:24:38.125639 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123698    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 03 23:24:38.125639 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123700    2577 feature_gate.go:328] unrecognized feature gate: Example
Jun 03 23:24:38.125639 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123703    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 03 23:24:38.125639 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123706    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 03 23:24:38.125639 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123708    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jun 03 23:24:38.125639 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123711    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 03 23:24:38.125639 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123713    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 03 23:24:38.125639 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123718    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 03 23:24:38.125639 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123721    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 03 23:24:38.125639 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123724    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 03 23:24:38.125639 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.123726    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 03 23:24:38.126031 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126018    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 03 23:24:38.126031 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126031    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126034    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126038    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126041    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126043    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126046    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126048    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126051    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126054    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126056    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126058    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126063    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126067    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126070    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126072    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126075    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126077    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126081    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126083    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126086    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 03 23:24:38.126087 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126088    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126091    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126094    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126098    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126101    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126103    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126106    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126109    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126111    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126114    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126116    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126119    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126121    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126124    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126126    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126129    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126131    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126134    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126137    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126139    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 03 23:24:38.126597 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126141    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126144    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126146    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126149    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126152    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126154    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126157    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126160    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126162    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126165    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126168    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126170    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126173    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126175    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126180    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126183    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126185    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126188    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126191    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 03 23:24:38.127095 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126193    2577 feature_gate.go:328] unrecognized feature gate: Example
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126197    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126201    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126217    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126221    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126224    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126227    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126230    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126232    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126235    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126239    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126242    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126244    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126247    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126250    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126252    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126255    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126257    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126260    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 03 23:24:38.127587 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126262    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126265    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126268    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126270    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126273    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126277    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126281    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126284    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126287    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126290    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126293    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126296    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126298    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126301    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126304    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126306    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126309    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126311    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126314    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126735    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jun 03 23:24:38.128056 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126741    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126745    2577 feature_gate.go:328] unrecognized feature gate: Example
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126748    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126751    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126753    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126756    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126759    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126761    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126764    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126767    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126769    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126773    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126777    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126779    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126782    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126785    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126787    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126791    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126793    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 03 23:24:38.128564 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126796    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126800    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126803    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126805    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126808    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126811    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126814    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126816    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126819    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126821    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126823    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126826    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126829    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126831    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126834    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126836    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126839    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126842    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126844    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126848    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 03 23:24:38.129026 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126851    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126854    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126857    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126860    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126862    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126864    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126867    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126870    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126873    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126875    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126878    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126881    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126884    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126887    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126889    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126893    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126908    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126912    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126915    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126917    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jun 03 23:24:38.129528 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126920    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126923    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126925    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126928    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126931    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126934    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126936    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126939    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126942    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126944    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126947    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126950    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126952    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126954    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126957    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126960    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126962    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126965    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126967    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126970    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jun 03 23:24:38.130070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126972    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126975    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126977    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126982    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126990    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126992    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126995    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.126997    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127000    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127003    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127005    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127008    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127010    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127013    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127015    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127020    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127023    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127026    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127088    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127095    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127102    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jun 03 23:24:38.130594 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127107    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127111    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127114    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127119    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127126    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127130    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127133    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127136    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127139    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127142    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127145    2577 flags.go:64] FLAG: --cgroup-root=""
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127148    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127152    2577 flags.go:64] FLAG: --client-ca-file=""
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127154    2577 flags.go:64] FLAG: --cloud-provider="external"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127157    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127162    2577 flags.go:64] FLAG: --cluster-domain=""
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127165    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127168    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127171    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127174    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127178    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127181    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127184    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jun 03 23:24:38.131114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127187    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127190    2577 flags.go:64] FLAG: --contention-profiling="false"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127193    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127196    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127199    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127216    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127221    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127223    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127226    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127229    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127232    2577 flags.go:64] FLAG: --enable-server="true"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127235    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127240    2577 flags.go:64] FLAG: --event-burst="100"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127242    2577 flags.go:64] FLAG: --event-qps="50"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127245    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127248    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127251    2577 flags.go:64] FLAG: --eviction-hard=""
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127255    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127257    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127260    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127263    2577 flags.go:64] FLAG: --eviction-soft=""
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127266    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127269    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127272    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127275    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jun 03 23:24:38.131689 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127278    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127281    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127284    2577 flags.go:64] FLAG: --feature-gates=""
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127287    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127291    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127294    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127297    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127300    2577 flags.go:64] FLAG: --healthz-port="10248"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127303    2577 flags.go:64] FLAG: --help="false"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127306    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127310    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127315    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127319    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127325    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127329    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127331    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127334    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127338    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127340    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127343    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127347    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127349    2577 flags.go:64] FLAG: --kube-reserved=""
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127352    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127355    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jun 03 23:24:38.132330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127358    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127361    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127364    2577 flags.go:64] FLAG: --lock-file=""
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127366    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127370    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127372    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127378    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127381    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127384    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127386    2577 flags.go:64] FLAG: --logging-format="text"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127390    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127393    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127396    2577 flags.go:64] FLAG: --manifest-url=""
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127399    2577 flags.go:64] FLAG: --manifest-url-header=""
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127403    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127406    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127410    2577 flags.go:64] FLAG: --max-pods="110"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127412    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127416    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127419    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127421    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127424    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127428    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127431    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127437    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jun 03 23:24:38.132925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127441    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127444    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127447    2577 flags.go:64] FLAG: --pod-cidr=""
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127450    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127455    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127458    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127461    2577 flags.go:64] FLAG: --pods-per-core="0"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127464    2577 flags.go:64] FLAG: --port="10250"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127467    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127470    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01afa1ceef06e74dc"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127473    2577 flags.go:64] FLAG: --qos-reserved=""
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127475    2577 flags.go:64] FLAG: --read-only-port="10255"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127478    2577 flags.go:64] FLAG: --register-node="true"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127481    2577 flags.go:64] FLAG: --register-with-taints=""
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127485    2577 flags.go:64] FLAG: --registry-burst="10"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127488    2577 flags.go:64] FLAG: --registry-qps="5"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127490    2577 flags.go:64] FLAG: --reserved-cpus=""
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127493    2577 flags.go:64] FLAG: --reserved-memory=""
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127497    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127500    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127503    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127506    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127509    2577 flags.go:64] FLAG: --runonce="false"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127511    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127515    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jun 03 23:24:38.133592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127518    2577 flags.go:64] FLAG: --seccomp-default="false"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127520    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127523    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127526    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127529    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127532    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127535    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127538    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127540    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127543    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127546    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127549    2577 flags.go:64] FLAG: --system-cgroups=""
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127551    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127557    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127559    2577 flags.go:64] FLAG: --tls-cert-file=""
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127562    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127566    2577 flags.go:64] FLAG: --tls-min-version=""
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127569    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127572    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127574    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127577    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127582    2577 flags.go:64] FLAG: --v="2"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127585    2577 flags.go:64] FLAG: --version="false"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127589    2577 flags.go:64] FLAG: --vmodule=""
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127593    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jun 03 23:24:38.134299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.127596    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127720    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127724    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127727    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127730    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127733    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127736    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127740    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127743    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127745    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127748    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127750    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127753    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127756    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127758    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127761    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127763    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127766    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127768    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127771    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 03 23:24:38.134926 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127774    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127776    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127779    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127782    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127784    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127787    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127789    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127792    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127796    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127799    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127801    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127805    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127809    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127812    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127816    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127819    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127822    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127824    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127827    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 03 23:24:38.135431 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127831    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127834    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127838    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127841    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127844    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127847    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127850    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127853    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127856    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127859    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127862    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127864    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127867    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127870    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127873    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127875    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127878    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127881    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127883    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 03 23:24:38.135902 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127886    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127889    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127892    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127895    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127898    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127900    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127903    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127906    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127908    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127911    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127914    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127917    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127919    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127923    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127926    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127930    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127933    2577 feature_gate.go:328] unrecognized feature gate: Example
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127936    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127938    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127941    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 03 23:24:38.136379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127943    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127946    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127948    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127951    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127953    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127956    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127959    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127961    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127964    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127966    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127969    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127971    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127974    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127977    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127981    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127984    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127986    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127989    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127991    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127994    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 03 23:24:38.136870 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.127996    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 03 23:24:38.137490 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.128534    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jun 03 23:24:38.137490 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.135223    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jun 03 23:24:38.137490 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.135239    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jun 03 23:24:38.137490 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.135260    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jun 03 23:24:38.137490 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.135266    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jun 03 23:24:38.137490 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135326    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 03 23:24:38.137490 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135331    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 03 23:24:38.137490 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135335    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 03 23:24:38.137490 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135338    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 03 23:24:38.137490 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135341    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jun 03 23:24:38.137490 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135344    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 03 23:24:38.137490 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135347    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 03 23:24:38.137490 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135350    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 03 23:24:38.137490 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135353    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 03 23:24:38.137490 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135355    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135358    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135360    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135363    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135367    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135371    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135374    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135377    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135380    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135383    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135386    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135389    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135392    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135395    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135397    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135400    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135403    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135405    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135408    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 03 23:24:38.137868 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135410    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135415    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135418    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135421    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135423    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135426    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135429    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135431    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135434    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135437    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135440    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135443    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135446    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135449    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135452    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135454    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135457    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135459    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135462    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135464    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 03 23:24:38.138361 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135467    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135469    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135472    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135474    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135477    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135480    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135482    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135485    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135487    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135490    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135493    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135495    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135498    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135501    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135504    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135507    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135510    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135512    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135516    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 03 23:24:38.138896 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135519    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135523    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135526    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135529    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135532    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135534    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135537    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135539    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135542    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135545    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135547    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135549    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135552    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135555    2577 feature_gate.go:328] unrecognized feature gate: Example
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135557    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135560    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135562    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135565    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135568    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135571    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 03 23:24:38.139418 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135575    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jun 03 23:24:38.140023 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135578    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 03 23:24:38.140023 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135580    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 03 23:24:38.140023 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135583    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 03 23:24:38.140023 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135586    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 03 23:24:38.140023 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135589    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 03 23:24:38.140023 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135592    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 03 23:24:38.140023 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135595    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 03 23:24:38.140023 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135598    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 03 23:24:38.140023 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135601    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 03 23:24:38.140023 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135604    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jun 03 23:24:38.140023 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.135609    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jun 03 23:24:38.140023 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135758    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jun 03 23:24:38.140023 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135763    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 03 23:24:38.140023 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135766    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 03 23:24:38.140023 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135769    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135772    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135775    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135778    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135780    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135783    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135786    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135789    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135791    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135794    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135797    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135799    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135802    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135806    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135809    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135812    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135815    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135817    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135820    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 03 23:24:38.140419 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135823    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135825    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135828    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135831    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135833    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135836    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135839    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135847    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135850    2577 feature_gate.go:328] unrecognized feature gate: Example
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135853    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135855    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135857    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135860    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135863    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135865    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135868    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135870    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135873    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135875    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135878    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jun 03 23:24:38.140973 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135880    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135883    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135885    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135888    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135890    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135893    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135896    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135898    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135902    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135905    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135908    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135911    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135914    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135917    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135921    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135925    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135927    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135930    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135933    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 03 23:24:38.141574 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135935    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135944    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135946    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135949    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135951    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135954    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135957    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135959    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135962    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135964    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135967    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135969    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135972    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135974    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135977    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135980    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135982    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135985    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135987    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135990    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 03 23:24:38.142379 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135992    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 03 23:24:38.143022 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135995    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 03 23:24:38.143022 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.135997    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 03 23:24:38.143022 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.136000    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 03 23:24:38.143022 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.136002    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 03 23:24:38.143022 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.136005    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 03 23:24:38.143022 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.136007    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 03 23:24:38.143022 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.136010    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jun 03 23:24:38.143022 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.136013    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 03 23:24:38.143022 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.136016    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 03 23:24:38.143022 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.136019    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jun 03 23:24:38.143022 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.136021    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 03 23:24:38.143022 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.136024    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jun 03 23:24:38.143022 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.136026    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 03 23:24:38.143022 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.136036    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 03 23:24:38.143022 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.136039    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 03 23:24:38.143022 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:38.136041    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 03 23:24:38.143518 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.136046    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jun 03 23:24:38.143518 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.136171    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jun 03 23:24:38.143518 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.138257    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jun 03 23:24:38.143518 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.139608    2577 server.go:1019] "Starting client certificate rotation"
Jun 03 23:24:38.143518 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.139734    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jun 03 23:24:38.143518 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.139783    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jun 03 23:24:38.143518 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.142302    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jun 03 23:24:38.148317 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.148254    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jun 03 23:24:38.160716 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.160699    2577 log.go:25] "Validated CRI v1 runtime API"
Jun 03 23:24:38.164244 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.164230    2577 log.go:25] "Validated CRI v1 image API"
Jun 03 23:24:38.165001 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.164984    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jun 03 23:24:38.167391 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.167373    2577 fs.go:135] Filesystem UUIDs: map[462b93ab-cb56-420b-a930-1d6f085d06bd:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9fa72e9a-d4cb-47b9-b843-994f3e51a3c3:/dev/nvme0n1p4]
Jun 03 23:24:38.167442 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.167391    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jun 03 23:24:38.169146 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.169130    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jun 03 23:24:38.172555 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.172446    2577 manager.go:216] Machine: {Timestamp:2026-06-03 23:24:38.171144382 +0000 UTC m=+0.333822447 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102671 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a423f08e3df4b3d6010119fcb6c4f SystemUUID:ec2a423f-08e3-df4b-3d60-10119fcb6c4f BootID:15e9cc07-104e-4ce9-b9c2-74e6c90fbcaa Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:91:cf:91:d8:c5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:91:cf:91:d8:c5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:96:64:a3:d8:2c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jun 03 23:24:38.172555 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.172547    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jun 03 23:24:38.172669 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.172621    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jun 03 23:24:38.175450 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.175429    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jun 03 23:24:38.175587 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.175453    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-110.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jun 03 23:24:38.176085 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.176076    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jun 03 23:24:38.176114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.176087    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jun 03 23:24:38.176114 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.176094    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jun 03 23:24:38.177054 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.177044    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jun 03 23:24:38.177809 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.177800    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jun 03 23:24:38.180581 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.180570    2577 state_mem.go:36] "Initialized new in-memory state store"
Jun 03 23:24:38.180683 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.180674    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jun 03 23:24:38.180712 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.180703    2577 kubelet.go:486] "Attempting to sync node with API server"
Jun 03 23:24:38.180745 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.180713    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jun 03 23:24:38.180745 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.180731    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jun 03 23:24:38.180745 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.180742    2577 kubelet.go:398] "Adding apiserver pod source"
Jun 03 23:24:38.180830 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.180752    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jun 03 23:24:38.181182 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.181170    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jun 03 23:24:38.181237 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.181191    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jun 03 23:24:38.183180 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.183164    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jun 03 23:24:38.184900 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.184883    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jun 03 23:24:38.184992 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.184915    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jun 03 23:24:38.185976 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.185965    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jun 03 23:24:38.186038 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.185980    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jun 03 23:24:38.186038 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.185987    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jun 03 23:24:38.186038 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.185993    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jun 03 23:24:38.186038 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.186003    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jun 03 23:24:38.186038 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.186009    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jun 03 23:24:38.186038 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.186017    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jun 03 23:24:38.186038 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.186023    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jun 03 23:24:38.186038 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.186030    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jun 03 23:24:38.186038 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.186036    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jun 03 23:24:38.186285 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.186050    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jun 03 23:24:38.186285 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.186055    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jun 03 23:24:38.186968 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.186958    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jun 03 23:24:38.186968 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.186967    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jun 03 23:24:38.190052 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.190035    2577 server.go:1268] "Started kubelet"
Jun 03 23:24:38.190160 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.190102    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jun 03 23:24:38.190160 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.190112    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jun 03 23:24:38.194242 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.194221    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jun 03 23:24:38.194666 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.194509    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-110.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 03 23:24:38.194666 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.194577    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-110.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jun 03 23:24:38.194796 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.194755    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jun 03 23:24:38.194938 ip-10-0-129-110 systemd[1]: Started Kubernetes Kubelet.
Jun 03 23:24:38.197170 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.197147    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jun 03 23:24:38.197597 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.197580    2577 server.go:310] "Adding debug handlers to kubelet server"
Jun 03 23:24:38.199257 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.198155    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-110.ec2.internal.18b5b5c351c4ea31  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-110.ec2.internal,UID:ip-10-0-129-110.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-110.ec2.internal,},FirstTimestamp:2026-06-03 23:24:38.190008881 +0000 UTC m=+0.352686946,LastTimestamp:2026-06-03 23:24:38.190008881 +0000 UTC m=+0.352686946,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-110.ec2.internal,}"
Jun 03 23:24:38.201421 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.201404    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jun 03 23:24:38.201631 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.201618    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jun 03 23:24:38.201684 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.201638    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jun 03 23:24:38.201743 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.201724    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-110.ec2.internal\" not found"
Jun 03 23:24:38.202313 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.202292    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jun 03 23:24:38.202313 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.202314    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jun 03 23:24:38.202454 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.202375    2577 reconstruct.go:97] "Volume reconstruction finished"
Jun 03 23:24:38.202454 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.202383    2577 reconciler.go:29] "Reconciler: start to sync state"
Jun 03 23:24:38.202454 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.202426    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jun 03 23:24:38.202701 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.202685    2577 factory.go:55] Registering systemd factory
Jun 03 23:24:38.202756 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.202705    2577 factory.go:223] Registration of the systemd container factory successfully
Jun 03 23:24:38.202907 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.202893    2577 factory.go:153] Registering CRI-O factory
Jun 03 23:24:38.202942 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.202913    2577 factory.go:223] Registration of the crio container factory successfully
Jun 03 23:24:38.202976 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.202967    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jun 03 23:24:38.203014 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.202992    2577 factory.go:103] Registering Raw factory
Jun 03 23:24:38.203014 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.203003    2577 manager.go:1196] Started watching for new ooms in manager
Jun 03 23:24:38.203346 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.203337    2577 manager.go:349] Starting recovery of all containers
Jun 03 23:24:38.211563 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.211399    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9wdl5"
Jun 03 23:24:38.211819 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.211774    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-110.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jun 03 23:24:38.213775 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.213760    2577 manager.go:354] Recovery completed
Jun 03 23:24:38.216168 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.216145    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jun 03 23:24:38.218013 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.218000    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jun 03 23:24:38.219608 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.219594    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasSufficientMemory"
Jun 03 23:24:38.219685 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.219624    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasNoDiskPressure"
Jun 03 23:24:38.219685 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.219638    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasSufficientPID"
Jun 03 23:24:38.220130 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.220114    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jun 03 23:24:38.220130 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.220127    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jun 03 23:24:38.220271 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.220129    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jun 03 23:24:38.220271 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.220154    2577 state_mem.go:36] "Initialized new in-memory state store"
Jun 03 23:24:38.221330 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.221315    2577 policy_none.go:49] "None policy: Start"
Jun 03 23:24:38.221376 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.221336    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jun 03 23:24:38.221376 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.221350    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jun 03 23:24:38.221477 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.221455    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasSufficientMemory"
Jun 03 23:24:38.221529 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.221488    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasNoDiskPressure"
Jun 03 23:24:38.221529 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.221504    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasSufficientPID"
Jun 03 23:24:38.222192 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.222124    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-110.ec2.internal.18b5b5c35388929c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-110.ec2.internal,UID:ip-10-0-129-110.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-110.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-110.ec2.internal,},FirstTimestamp:2026-06-03 23:24:38.219608732 +0000 UTC m=+0.382286803,LastTimestamp:2026-06-03 23:24:38.219608732 +0000 UTC m=+0.382286803,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-110.ec2.internal,}"
Jun 03 23:24:38.222287 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.222275    2577 policy_none.go:47] "Start"
Jun 03 23:24:38.223851 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.223832    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9wdl5"
Jun 03 23:24:38.252857 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.252840    2577 manager.go:337] "Starting Device Plugin manager"
Jun 03 23:24:38.252962 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.252883    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jun 03 23:24:38.252962 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.252896    2577 server.go:87] "Starting device plugin registration server"
Jun 03 23:24:38.253196 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.253182    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jun 03 23:24:38.253276 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.253197    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jun 03 23:24:38.253320 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.253281    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jun 03 23:24:38.253416 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.253389    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jun 03 23:24:38.253416 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.253404    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jun 03 23:24:38.253907 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.253888    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jun 03 23:24:38.253967 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.253943    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-110.ec2.internal\" not found"
Jun 03 23:24:38.271240 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.271183    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jun 03 23:24:38.275656 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.272300    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jun 03 23:24:38.275656 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.272332    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jun 03 23:24:38.275656 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.272350    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jun 03 23:24:38.275656 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.272381    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jun 03 23:24:38.275656 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.274573    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 03 23:24:38.353664 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.353638    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jun 03 23:24:38.354583 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.354559    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasSufficientMemory"
Jun 03 23:24:38.354693 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.354594    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasNoDiskPressure"
Jun 03 23:24:38.354693 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.354610    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasSufficientPID"
Jun 03 23:24:38.354693 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.354641    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.367625 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.367546    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.367625 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.367572    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-110.ec2.internal\": node \"ip-10-0-129-110.ec2.internal\" not found"
Jun 03 23:24:38.372736 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.372720    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-110.ec2.internal"]
Jun 03 23:24:38.372802 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.372780    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jun 03 23:24:38.373573 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.373551    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasSufficientMemory"
Jun 03 23:24:38.373654 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.373579    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasNoDiskPressure"
Jun 03 23:24:38.373654 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.373595    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasSufficientPID"
Jun 03 23:24:38.375025 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.375010    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jun 03 23:24:38.375108 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.375090    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.375157 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.375116    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jun 03 23:24:38.375668 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.375652    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasSufficientMemory"
Jun 03 23:24:38.375668 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.375660    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasSufficientMemory"
Jun 03 23:24:38.375798 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.375679    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasNoDiskPressure"
Jun 03 23:24:38.375798 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.375684    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasNoDiskPressure"
Jun 03 23:24:38.375798 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.375688    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasSufficientPID"
Jun 03 23:24:38.375798 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.375697    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasSufficientPID"
Jun 03 23:24:38.376951 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.376934    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.376994 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.376968    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jun 03 23:24:38.377598 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.377582    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasSufficientMemory"
Jun 03 23:24:38.377651 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.377608    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasNoDiskPressure"
Jun 03 23:24:38.377651 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.377619    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeHasSufficientPID"
Jun 03 23:24:38.388306 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.388291    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-110.ec2.internal\" not found"
Jun 03 23:24:38.394762 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.394745    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-110.ec2.internal\" not found" node="ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.398555 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.398539    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-110.ec2.internal\" not found" node="ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.488408 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.488382    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-110.ec2.internal\" not found"
Jun 03 23:24:38.503556 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.503530    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/da2bd6e93c11d61b803364c8204bbd7a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal\" (UID: \"da2bd6e93c11d61b803364c8204bbd7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.503616 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.503560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da2bd6e93c11d61b803364c8204bbd7a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal\" (UID: \"da2bd6e93c11d61b803364c8204bbd7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.503616 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.503578    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a14572b805eb35459a9a5413dc15e576-config\") pod \"kube-apiserver-proxy-ip-10-0-129-110.ec2.internal\" (UID: \"a14572b805eb35459a9a5413dc15e576\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.588517 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.588491    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-110.ec2.internal\" not found"
Jun 03 23:24:38.603776 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.603752    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a14572b805eb35459a9a5413dc15e576-config\") pod \"kube-apiserver-proxy-ip-10-0-129-110.ec2.internal\" (UID: \"a14572b805eb35459a9a5413dc15e576\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.603826 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.603779    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/da2bd6e93c11d61b803364c8204bbd7a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal\" (UID: \"da2bd6e93c11d61b803364c8204bbd7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.603826 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.603799    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da2bd6e93c11d61b803364c8204bbd7a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal\" (UID: \"da2bd6e93c11d61b803364c8204bbd7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.603886 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.603858    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a14572b805eb35459a9a5413dc15e576-config\") pod \"kube-apiserver-proxy-ip-10-0-129-110.ec2.internal\" (UID: \"a14572b805eb35459a9a5413dc15e576\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.603941 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.603925    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da2bd6e93c11d61b803364c8204bbd7a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal\" (UID: \"da2bd6e93c11d61b803364c8204bbd7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.603976 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.603968    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/da2bd6e93c11d61b803364c8204bbd7a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal\" (UID: \"da2bd6e93c11d61b803364c8204bbd7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.689181 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.689101    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-110.ec2.internal\" not found"
Jun 03 23:24:38.697272 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.697251    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.701735 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:38.701716    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-110.ec2.internal"
Jun 03 23:24:38.789868 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.789836    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-110.ec2.internal\" not found"
Jun 03 23:24:38.890346 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.890316    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-110.ec2.internal\" not found"
Jun 03 23:24:38.990848 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:38.990777    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-110.ec2.internal\" not found"
Jun 03 23:24:39.043996 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.043961    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 03 23:24:39.102550 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.102524    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal"
Jun 03 23:24:39.117007 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.116976    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jun 03 23:24:39.119418 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.119396    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-110.ec2.internal"
Jun 03 23:24:39.129382 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.129365    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jun 03 23:24:39.139468 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.139446    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jun 03 23:24:39.139612 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.139593    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jun 03 23:24:39.139664 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.139620    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jun 03 23:24:39.169394 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:39.169363    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda2bd6e93c11d61b803364c8204bbd7a.slice/crio-ef64ecf970a8a0508d13d57dbe14eac483b774f2aa35092b987b5fc002bec31b WatchSource:0}: Error finding container ef64ecf970a8a0508d13d57dbe14eac483b774f2aa35092b987b5fc002bec31b: Status 404 returned error can't find the container with id ef64ecf970a8a0508d13d57dbe14eac483b774f2aa35092b987b5fc002bec31b
Jun 03 23:24:39.169746 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:39.169721    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda14572b805eb35459a9a5413dc15e576.slice/crio-4e412ce8a5f04052f2f469ddad01931379b56a7ca4a12eaac459649a8e981e16 WatchSource:0}: Error finding container 4e412ce8a5f04052f2f469ddad01931379b56a7ca4a12eaac459649a8e981e16: Status 404 returned error can't find the container with id 4e412ce8a5f04052f2f469ddad01931379b56a7ca4a12eaac459649a8e981e16
Jun 03 23:24:39.175126 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.175112    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 03 23:24:39.180909 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.180892    2577 apiserver.go:52] "Watching apiserver"
Jun 03 23:24:39.190356 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.190338    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jun 03 23:24:39.192526 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.192500    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-n99j6","openshift-multus/multus-additional-cni-plugins-rgddf","openshift-multus/multus-rkg7d","openshift-image-registry/node-ca-6n76l","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal","openshift-multus/network-metrics-daemon-ssbpc","openshift-network-diagnostics/network-check-target-9gpsw","openshift-network-operator/iptables-alerter-ll5ls","openshift-ovn-kubernetes/ovnkube-node-4s5gr","kube-system/global-pull-secret-syncer-9d2cb","kube-system/konnectivity-agent-kx2mq","kube-system/kube-apiserver-proxy-ip-10-0-129-110.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl","openshift-cluster-node-tuning-operator/tuned-bf9r5"]
Jun 03 23:24:39.195670 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.195650    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:39.195743 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.195714    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:24:39.196690 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.196674    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kx2mq"
Jun 03 23:24:39.197724 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.197706    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.198844 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.198823    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jun 03 23:24:39.198924 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.198869    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7zqwp\""
Jun 03 23:24:39.198924 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.198873    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jun 03 23:24:39.199026 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.198934    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6n76l"
Jun 03 23:24:39.199454 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.199438    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jun 03 23:24:39.199573 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.199555    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jun 03 23:24:39.199753 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.199736    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jun 03 23:24:39.199825 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.199807    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jun 03 23:24:39.199865 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.199827    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kz25d\""
Jun 03 23:24:39.200105 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.200087    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.200752 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.200732    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jun 03 23:24:39.200836 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.200754    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jun 03 23:24:39.200981 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.200968    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jun 03 23:24:39.201234 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.201220    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-c7766\""
Jun 03 23:24:39.201773 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.201759    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jun 03 23:24:39.201893 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.201873    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:39.201980 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.201955    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:24:39.202300 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.202281    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jun 03 23:24:39.202544 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.202525    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jun 03 23:24:39.202859 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.202846    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-b22qj\""
Jun 03 23:24:39.203802 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.203383    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.204658 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.204643    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.205538 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.205521    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jun 03 23:24:39.205835 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.205817    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jun 03 23:24:39.205889 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.205837    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-multus-conf-dir\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.205931 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.205888    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-lib-modules\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.205931 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.205912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-kubelet\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.206017 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.205936    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-run-ovn-kubernetes\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.206017 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.205960    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-cni-netd\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.206017 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206004    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-systemd-units\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.206159 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206037    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-cnibin\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.206159 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-run-netns\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.206159 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206091    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-sysconfig\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.206159 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206095    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.206159 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206114    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-var-lib-kubelet\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.206159 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206138    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b70ba3a5-03df-4847-9f88-12c8ae56099c-serviceca\") pod \"node-ca-6n76l\" (UID: \"b70ba3a5-03df-4847-9f88-12c8ae56099c\") " pod="openshift-image-registry/node-ca-6n76l"
Jun 03 23:24:39.206471 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206165    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ffcs\" (UniqueName: \"kubernetes.io/projected/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-kube-api-access-8ffcs\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.206471 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206190    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-slash\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.206471 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206229    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-cni-bin\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.206471 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206253    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f0a1234d-160e-4994-834f-b87872953ed1-konnectivity-ca\") pod \"konnectivity-agent-kx2mq\" (UID: \"f0a1234d-160e-4994-834f-b87872953ed1\") " pod="kube-system/konnectivity-agent-kx2mq"
Jun 03 23:24:39.206471 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206260    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jun 03 23:24:39.206471 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206296    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-sys\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.206471 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206330    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-kubernetes\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.206471 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-sysctl-conf\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.206471 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206414    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-run-openvswitch\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.206471 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206440    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-multus-cni-dir\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.206471 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206467    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-run-k8s-cni-cncf-io\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206496    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-ovnkube-script-lib\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206519    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-tmp\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206549    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-run-netns\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206577    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-os-release\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-systemd\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206622    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-host\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206647    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-multus-socket-dir-parent\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206682    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-var-lib-kubelet\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206721    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-run-systemd\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206732    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206745    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-etc-openvswitch\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206786    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-env-overrides\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206812    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-ovn-node-metrics-cert\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dggl\" (UniqueName: \"kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl\") pod \"network-check-target-9gpsw\" (UID: \"7ab2d105-78a7-4089-9f3b-f0b03411ec50\") " pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206860    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-var-lib-cni-multus\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206886    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-run-ovn\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.206899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206901    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-system-cni-dir\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206917    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-etc-kubernetes\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206951    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwz7z\" (UniqueName: \"kubernetes.io/projected/06e69283-7ec4-4312-8f28-dc75d920d672-kube-api-access-nwz7z\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.206990    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06e69283-7ec4-4312-8f28-dc75d920d672-cni-binary-copy\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-tuned\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207037    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-js8gq\""
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207039    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207074    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-722lb\" (UniqueName: \"kubernetes.io/projected/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-kube-api-access-722lb\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207101    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cx4x\" (UniqueName: \"kubernetes.io/projected/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-kube-api-access-6cx4x\") pod \"network-metrics-daemon-ssbpc\" (UID: \"81ee1ff9-1018-4eaf-b246-18b5f7a95f78\") " pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207115    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6xlz7\""
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207129    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-var-lib-cni-bin\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207138    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207151    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-run-multus-certs\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207141    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207173    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-modprobe-d\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207198    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs\") pod \"network-metrics-daemon-ssbpc\" (UID: \"81ee1ff9-1018-4eaf-b246-18b5f7a95f78\") " pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207237    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f0a1234d-160e-4994-834f-b87872953ed1-agent-certs\") pod \"konnectivity-agent-kx2mq\" (UID: \"f0a1234d-160e-4994-834f-b87872953ed1\") " pod="kube-system/konnectivity-agent-kx2mq"
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207257    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207292    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sdrk\" (UniqueName: \"kubernetes.io/projected/b70ba3a5-03df-4847-9f88-12c8ae56099c-kube-api-access-2sdrk\") pod \"node-ca-6n76l\" (UID: \"b70ba3a5-03df-4847-9f88-12c8ae56099c\") " pod="openshift-image-registry/node-ca-6n76l"
Jun 03 23:24:39.207636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207320    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-sysctl-d\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.208238 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207363    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-run\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.208238 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207399    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-node-log\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.208238 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207433    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-ovnkube-config\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.208238 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207476    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06e69283-7ec4-4312-8f28-dc75d920d672-multus-daemon-config\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.208238 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207526    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b70ba3a5-03df-4847-9f88-12c8ae56099c-host\") pod \"node-ca-6n76l\" (UID: \"b70ba3a5-03df-4847-9f88-12c8ae56099c\") " pod="openshift-image-registry/node-ca-6n76l"
Jun 03 23:24:39.208238 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207545    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jun 03 23:24:39.208238 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207553    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-var-lib-openvswitch\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.208238 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207584    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:39.208238 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207576    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-log-socket\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.208238 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.207638    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-hostroot\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.208238 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.207662    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:24:39.208551 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.208537    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jun 03 23:24:39.208978 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.208963    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n99j6"
Jun 03 23:24:39.209783 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.209762    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jun 03 23:24:39.209869 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.209809    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jun 03 23:24:39.209869 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.209812    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xl7gn\""
Jun 03 23:24:39.210642 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.210623    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ll5ls"
Jun 03 23:24:39.211122 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.211106    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jun 03 23:24:39.211197 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.211138    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jun 03 23:24:39.211197 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.211154    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5rl5k\""
Jun 03 23:24:39.212596 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.212572    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jun 03 23:24:39.212697 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.212658    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6zqk2\""
Jun 03 23:24:39.212760 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.212736    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jun 03 23:24:39.214248 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.213960    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jun 03 23:24:39.216954 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.216936    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jun 03 23:24:39.226079 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.226044    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-02 23:19:38 +0000 UTC" deadline="2027-12-15 04:27:59.189042574 +0000 UTC"
Jun 03 23:24:39.226079 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.226078    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13421h3m19.962966304s"
Jun 03 23:24:39.237909 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.237893    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-z9nxz"
Jun 03 23:24:39.244836 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.244819    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-z9nxz"
Jun 03 23:24:39.275065 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.275024    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal" event={"ID":"da2bd6e93c11d61b803364c8204bbd7a","Type":"ContainerStarted","Data":"ef64ecf970a8a0508d13d57dbe14eac483b774f2aa35092b987b5fc002bec31b"}
Jun 03 23:24:39.276389 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.276370    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-110.ec2.internal" event={"ID":"a14572b805eb35459a9a5413dc15e576","Type":"ContainerStarted","Data":"4e412ce8a5f04052f2f469ddad01931379b56a7ca4a12eaac459649a8e981e16"}
Jun 03 23:24:39.303961 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.303938    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jun 03 23:24:39.308608 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308591    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-ovnkube-config\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.308677 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308618    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.308677 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308635    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46xl9\" (UniqueName: \"kubernetes.io/projected/76222ee2-ede8-4c5b-88ca-6da4400d27da-kube-api-access-46xl9\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.308677 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06e69283-7ec4-4312-8f28-dc75d920d672-multus-daemon-config\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.308677 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308673    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b70ba3a5-03df-4847-9f88-12c8ae56099c-host\") pod \"node-ca-6n76l\" (UID: \"b70ba3a5-03df-4847-9f88-12c8ae56099c\") " pod="openshift-image-registry/node-ca-6n76l"
Jun 03 23:24:39.308857 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308710    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-var-lib-openvswitch\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.308857 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308735    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b70ba3a5-03df-4847-9f88-12c8ae56099c-host\") pod \"node-ca-6n76l\" (UID: \"b70ba3a5-03df-4847-9f88-12c8ae56099c\") " pod="openshift-image-registry/node-ca-6n76l"
Jun 03 23:24:39.308857 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308765    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-log-socket\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.308857 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308772    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-var-lib-openvswitch\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.309046 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308865    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29474ba6-d997-4d27-a609-7adc008c79ce-system-cni-dir\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.309046 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308870    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-log-socket\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.309046 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308901    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/29474ba6-d997-4d27-a609-7adc008c79ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.309046 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308919    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret\") pod \"global-pull-secret-syncer-9d2cb\" (UID: \"55ceda9f-6200-4a21-a050-d290d631043f\") " pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:39.309046 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308948    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd-host-slash\") pod \"iptables-alerter-ll5ls\" (UID: \"bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd\") " pod="openshift-network-operator/iptables-alerter-ll5ls"
Jun 03 23:24:39.309046 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308968    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-hostroot\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.309046 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-multus-conf-dir\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.309046 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.308999    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-lib-modules\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.309046 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-kubelet\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-run-ovn-kubernetes\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309084    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-cni-netd\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309063    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-hostroot\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309105    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-device-dir\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309107    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-kubelet\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309103    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-run-ovn-kubernetes\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309148    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-multus-conf-dir\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309158    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-lib-modules\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309188    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-cni-netd\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309226    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/55ceda9f-6200-4a21-a050-d290d631043f-kubelet-config\") pod \"global-pull-secret-syncer-9d2cb\" (UID: \"55ceda9f-6200-4a21-a050-d290d631043f\") " pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309255    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-systemd-units\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309272    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29474ba6-d997-4d27-a609-7adc008c79ce-os-release\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309281    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-systemd-units\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309296    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-cnibin\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309313    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-run-netns\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309328    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-sysconfig\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.309638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309332    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-ovnkube-config\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309347    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l95dz\" (UniqueName: \"kubernetes.io/projected/bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd-kube-api-access-l95dz\") pod \"iptables-alerter-ll5ls\" (UID: \"bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd\") " pod="openshift-network-operator/iptables-alerter-ll5ls"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309372    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-var-lib-kubelet\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309385    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-run-netns\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309402    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-sysconfig\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b70ba3a5-03df-4847-9f88-12c8ae56099c-serviceca\") pod \"node-ca-6n76l\" (UID: \"b70ba3a5-03df-4847-9f88-12c8ae56099c\") " pod="openshift-image-registry/node-ca-6n76l"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309422    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-var-lib-kubelet\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309421    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-cnibin\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309452    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ffcs\" (UniqueName: \"kubernetes.io/projected/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-kube-api-access-8ffcs\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309511    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-slash\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-cni-bin\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309550    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f0a1234d-160e-4994-834f-b87872953ed1-konnectivity-ca\") pod \"konnectivity-agent-kx2mq\" (UID: \"f0a1234d-160e-4994-834f-b87872953ed1\") " pod="kube-system/konnectivity-agent-kx2mq"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309552    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-slash\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309579    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-cni-bin\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309643    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-sys\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309739    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-kubernetes\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309769    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-sys\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-sysctl-conf\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.310343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309833    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-kubernetes\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309837    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b70ba3a5-03df-4847-9f88-12c8ae56099c-serviceca\") pod \"node-ca-6n76l\" (UID: \"b70ba3a5-03df-4847-9f88-12c8ae56099c\") " pod="openshift-image-registry/node-ca-6n76l"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309860    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-run-openvswitch\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309890    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-socket-dir\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309910    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-run-openvswitch\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309933    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06e69283-7ec4-4312-8f28-dc75d920d672-multus-daemon-config\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309956    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-sys-fs\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309972    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-sysctl-conf\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.309996    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29474ba6-d997-4d27-a609-7adc008c79ce-cnibin\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/29474ba6-d997-4d27-a609-7adc008c79ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310031    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-multus-cni-dir\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310049    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-run-k8s-cni-cncf-io\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310064    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-ovnkube-script-lib\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-tmp\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310093    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-run-netns\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310092    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-multus-cni-dir\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-registration-dir\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.311131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310108    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f0a1234d-160e-4994-834f-b87872953ed1-konnectivity-ca\") pod \"konnectivity-agent-kx2mq\" (UID: \"f0a1234d-160e-4994-834f-b87872953ed1\") " pod="kube-system/konnectivity-agent-kx2mq"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310144    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-run-k8s-cni-cncf-io\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310151    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-run-netns\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310131    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-etc-selinux\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310237    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77w88\" (UniqueName: \"kubernetes.io/projected/29474ba6-d997-4d27-a609-7adc008c79ce-kube-api-access-77w88\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310281    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srbz6\" (UniqueName: \"kubernetes.io/projected/e42566fd-c3a2-48b7-8d03-a172d908aeff-kube-api-access-srbz6\") pod \"node-resolver-n99j6\" (UID: \"e42566fd-c3a2-48b7-8d03-a172d908aeff\") " pod="openshift-dns/node-resolver-n99j6"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-os-release\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-systemd\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310353    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-host\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310382    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29474ba6-d997-4d27-a609-7adc008c79ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-multus-socket-dir-parent\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310407    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-systemd\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310424    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-var-lib-kubelet\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310474    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-run-systemd\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310489    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-var-lib-kubelet\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310497    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-os-release\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310511    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-etc-openvswitch\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.311925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310549    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-etc-openvswitch\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310542    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-run-systemd\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310564    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-host\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310551    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-env-overrides\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310583    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-ovnkube-script-lib\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310600    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-ovn-node-metrics-cert\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310635    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dggl\" (UniqueName: \"kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl\") pod \"network-check-target-9gpsw\" (UID: \"7ab2d105-78a7-4089-9f3b-f0b03411ec50\") " pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310652    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-var-lib-cni-multus\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310674    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-run-ovn\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310699    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29474ba6-d997-4d27-a609-7adc008c79ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310705    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-var-lib-cni-multus\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310726    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e42566fd-c3a2-48b7-8d03-a172d908aeff-hosts-file\") pod \"node-resolver-n99j6\" (UID: \"e42566fd-c3a2-48b7-8d03-a172d908aeff\") " pod="openshift-dns/node-resolver-n99j6"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310738    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-run-ovn\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310751    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e42566fd-c3a2-48b7-8d03-a172d908aeff-tmp-dir\") pod \"node-resolver-n99j6\" (UID: \"e42566fd-c3a2-48b7-8d03-a172d908aeff\") " pod="openshift-dns/node-resolver-n99j6"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-system-cni-dir\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310798    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-etc-kubernetes\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310823    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwz7z\" (UniqueName: \"kubernetes.io/projected/06e69283-7ec4-4312-8f28-dc75d920d672-kube-api-access-nwz7z\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.312723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310844    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/55ceda9f-6200-4a21-a050-d290d631043f-dbus\") pod \"global-pull-secret-syncer-9d2cb\" (UID: \"55ceda9f-6200-4a21-a050-d290d631043f\") " pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310849    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-etc-kubernetes\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310857    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-system-cni-dir\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310868    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06e69283-7ec4-4312-8f28-dc75d920d672-cni-binary-copy\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.310930    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-env-overrides\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-tuned\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311041    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-multus-socket-dir-parent\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311053    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311075    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-722lb\" (UniqueName: \"kubernetes.io/projected/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-kube-api-access-722lb\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cx4x\" (UniqueName: \"kubernetes.io/projected/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-kube-api-access-6cx4x\") pod \"network-metrics-daemon-ssbpc\" (UID: \"81ee1ff9-1018-4eaf-b246-18b5f7a95f78\") " pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311125    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-var-lib-cni-bin\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311157    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-var-lib-cni-bin\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-run-multus-certs\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311221    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-modprobe-d\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311268    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs\") pod \"network-metrics-daemon-ssbpc\" (UID: \"81ee1ff9-1018-4eaf-b246-18b5f7a95f78\") " pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311302    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd-iptables-alerter-script\") pod \"iptables-alerter-ll5ls\" (UID: \"bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd\") " pod="openshift-network-operator/iptables-alerter-ll5ls"
Jun 03 23:24:39.313575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311316    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06e69283-7ec4-4312-8f28-dc75d920d672-host-run-multus-certs\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.314162 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311329    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f0a1234d-160e-4994-834f-b87872953ed1-agent-certs\") pod \"konnectivity-agent-kx2mq\" (UID: \"f0a1234d-160e-4994-834f-b87872953ed1\") " pod="kube-system/konnectivity-agent-kx2mq"
Jun 03 23:24:39.314162 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311348    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06e69283-7ec4-4312-8f28-dc75d920d672-cni-binary-copy\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.314162 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sdrk\" (UniqueName: \"kubernetes.io/projected/b70ba3a5-03df-4847-9f88-12c8ae56099c-kube-api-access-2sdrk\") pod \"node-ca-6n76l\" (UID: \"b70ba3a5-03df-4847-9f88-12c8ae56099c\") " pod="openshift-image-registry/node-ca-6n76l"
Jun 03 23:24:39.314162 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311389    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-sysctl-d\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.314162 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-run\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.314162 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311429    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-modprobe-d\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.314162 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311436    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-node-log\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.314162 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311505    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-run\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.314162 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311507    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-sysctl-d\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.314162 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.311508    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-node-log\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.314162 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.311658    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 23:24:39.314162 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.311725    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs podName:81ee1ff9-1018-4eaf-b246-18b5f7a95f78 nodeName:}" failed. No retries permitted until 2026-06-03 23:24:39.81170813 +0000 UTC m=+1.974386201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs") pod "network-metrics-daemon-ssbpc" (UID: "81ee1ff9-1018-4eaf-b246-18b5f7a95f78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 23:24:39.314162 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.313920    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-tmp\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.314162 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.313963    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-etc-tuned\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.314162 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.314111    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-ovn-node-metrics-cert\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.314648 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.314308    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f0a1234d-160e-4994-834f-b87872953ed1-agent-certs\") pod \"konnectivity-agent-kx2mq\" (UID: \"f0a1234d-160e-4994-834f-b87872953ed1\") " pod="kube-system/konnectivity-agent-kx2mq"
Jun 03 23:24:39.318478 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.318462    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 03 23:24:39.318478 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.318479    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 03 23:24:39.318582 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.318487    2577 projected.go:196] Error preparing data for projected volume kube-api-access-6dggl for pod openshift-network-diagnostics/network-check-target-9gpsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 23:24:39.318582 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.318541    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ffcs\" (UniqueName: \"kubernetes.io/projected/dff5f4ce-7ddd-4c84-9b4a-0170acb8c493-kube-api-access-8ffcs\") pod \"tuned-bf9r5\" (UID: \"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493\") " pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.318642 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.318594    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl podName:7ab2d105-78a7-4089-9f3b-f0b03411ec50 nodeName:}" failed. No retries permitted until 2026-06-03 23:24:39.818580041 +0000 UTC m=+1.981258097 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6dggl" (UniqueName: "kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl") pod "network-check-target-9gpsw" (UID: "7ab2d105-78a7-4089-9f3b-f0b03411ec50") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 23:24:39.323452 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.323426    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cx4x\" (UniqueName: \"kubernetes.io/projected/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-kube-api-access-6cx4x\") pod \"network-metrics-daemon-ssbpc\" (UID: \"81ee1ff9-1018-4eaf-b246-18b5f7a95f78\") " pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:39.323984 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.323957    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwz7z\" (UniqueName: \"kubernetes.io/projected/06e69283-7ec4-4312-8f28-dc75d920d672-kube-api-access-nwz7z\") pod \"multus-rkg7d\" (UID: \"06e69283-7ec4-4312-8f28-dc75d920d672\") " pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.324332 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.324313    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sdrk\" (UniqueName: \"kubernetes.io/projected/b70ba3a5-03df-4847-9f88-12c8ae56099c-kube-api-access-2sdrk\") pod \"node-ca-6n76l\" (UID: \"b70ba3a5-03df-4847-9f88-12c8ae56099c\") " pod="openshift-image-registry/node-ca-6n76l"
Jun 03 23:24:39.326355 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.326334    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-722lb\" (UniqueName: \"kubernetes.io/projected/60fa5a57-9316-4b22-a1ca-c5f40a4bc065-kube-api-access-722lb\") pod \"ovnkube-node-4s5gr\" (UID: \"60fa5a57-9316-4b22-a1ca-c5f40a4bc065\") " pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.398510 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.398479    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 03 23:24:39.411736 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.411706    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29474ba6-d997-4d27-a609-7adc008c79ce-system-cni-dir\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.411856 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.411750    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/29474ba6-d997-4d27-a609-7adc008c79ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.411856 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.411765    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29474ba6-d997-4d27-a609-7adc008c79ce-system-cni-dir\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.411856 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.411776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret\") pod \"global-pull-secret-syncer-9d2cb\" (UID: \"55ceda9f-6200-4a21-a050-d290d631043f\") " pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:39.411856 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.411808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd-host-slash\") pod \"iptables-alerter-ll5ls\" (UID: \"bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd\") " pod="openshift-network-operator/iptables-alerter-ll5ls"
Jun 03 23:24:39.411856 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.411852    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd-host-slash\") pod \"iptables-alerter-ll5ls\" (UID: \"bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd\") " pod="openshift-network-operator/iptables-alerter-ll5ls"
Jun 03 23:24:39.412074 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.411857    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 03 23:24:39.412074 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.411877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-device-dir\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.412074 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.411906    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/55ceda9f-6200-4a21-a050-d290d631043f-kubelet-config\") pod \"global-pull-secret-syncer-9d2cb\" (UID: \"55ceda9f-6200-4a21-a050-d290d631043f\") " pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:39.412074 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.411928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29474ba6-d997-4d27-a609-7adc008c79ce-os-release\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.412074 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.411955    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l95dz\" (UniqueName: \"kubernetes.io/projected/bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd-kube-api-access-l95dz\") pod \"iptables-alerter-ll5ls\" (UID: \"bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd\") " pod="openshift-network-operator/iptables-alerter-ll5ls"
Jun 03 23:24:39.412074 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.411988    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-socket-dir\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.412074 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412011    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-sys-fs\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.412074 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.412048    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret podName:55ceda9f-6200-4a21-a050-d290d631043f nodeName:}" failed. No retries permitted until 2026-06-03 23:24:39.912014272 +0000 UTC m=+2.074692334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret") pod "global-pull-secret-syncer-9d2cb" (UID: "55ceda9f-6200-4a21-a050-d290d631043f") : object "kube-system"/"original-pull-secret" not registered
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412082    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/55ceda9f-6200-4a21-a050-d290d631043f-kubelet-config\") pod \"global-pull-secret-syncer-9d2cb\" (UID: \"55ceda9f-6200-4a21-a050-d290d631043f\") " pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412089    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29474ba6-d997-4d27-a609-7adc008c79ce-cnibin\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412123    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29474ba6-d997-4d27-a609-7adc008c79ce-cnibin\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412044    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-device-dir\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412138    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/29474ba6-d997-4d27-a609-7adc008c79ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412153    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-sys-fs\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412172    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-registration-dir\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-etc-selinux\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412236    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77w88\" (UniqueName: \"kubernetes.io/projected/29474ba6-d997-4d27-a609-7adc008c79ce-kube-api-access-77w88\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412259    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srbz6\" (UniqueName: \"kubernetes.io/projected/e42566fd-c3a2-48b7-8d03-a172d908aeff-kube-api-access-srbz6\") pod \"node-resolver-n99j6\" (UID: \"e42566fd-c3a2-48b7-8d03-a172d908aeff\") " pod="openshift-dns/node-resolver-n99j6"
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412268    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29474ba6-d997-4d27-a609-7adc008c79ce-os-release\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29474ba6-d997-4d27-a609-7adc008c79ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412318    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-socket-dir\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29474ba6-d997-4d27-a609-7adc008c79ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412333    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-registration-dir\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412352    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e42566fd-c3a2-48b7-8d03-a172d908aeff-hosts-file\") pod \"node-resolver-n99j6\" (UID: \"e42566fd-c3a2-48b7-8d03-a172d908aeff\") " pod="openshift-dns/node-resolver-n99j6"
Jun 03 23:24:39.412524 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e42566fd-c3a2-48b7-8d03-a172d908aeff-tmp-dir\") pod \"node-resolver-n99j6\" (UID: \"e42566fd-c3a2-48b7-8d03-a172d908aeff\") " pod="openshift-dns/node-resolver-n99j6"
Jun 03 23:24:39.413108 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412401    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/55ceda9f-6200-4a21-a050-d290d631043f-dbus\") pod \"global-pull-secret-syncer-9d2cb\" (UID: \"55ceda9f-6200-4a21-a050-d290d631043f\") " pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:39.413108 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412411    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/29474ba6-d997-4d27-a609-7adc008c79ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.413108 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412443    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd-iptables-alerter-script\") pod \"iptables-alerter-ll5ls\" (UID: \"bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd\") " pod="openshift-network-operator/iptables-alerter-ll5ls"
Jun 03 23:24:39.413108 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412460    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-etc-selinux\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.413108 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412473    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.413108 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412484    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e42566fd-c3a2-48b7-8d03-a172d908aeff-hosts-file\") pod \"node-resolver-n99j6\" (UID: \"e42566fd-c3a2-48b7-8d03-a172d908aeff\") " pod="openshift-dns/node-resolver-n99j6"
Jun 03 23:24:39.413108 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412498    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46xl9\" (UniqueName: \"kubernetes.io/projected/76222ee2-ede8-4c5b-88ca-6da4400d27da-kube-api-access-46xl9\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.413108 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412498    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29474ba6-d997-4d27-a609-7adc008c79ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.413108 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412545    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/55ceda9f-6200-4a21-a050-d290d631043f-dbus\") pod \"global-pull-secret-syncer-9d2cb\" (UID: \"55ceda9f-6200-4a21-a050-d290d631043f\") " pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:39.413108 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412563    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/76222ee2-ede8-4c5b-88ca-6da4400d27da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.413108 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412709    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/29474ba6-d997-4d27-a609-7adc008c79ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.413108 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412777    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e42566fd-c3a2-48b7-8d03-a172d908aeff-tmp-dir\") pod \"node-resolver-n99j6\" (UID: \"e42566fd-c3a2-48b7-8d03-a172d908aeff\") " pod="openshift-dns/node-resolver-n99j6"
Jun 03 23:24:39.413108 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.412957    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd-iptables-alerter-script\") pod \"iptables-alerter-ll5ls\" (UID: \"bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd\") " pod="openshift-network-operator/iptables-alerter-ll5ls"
Jun 03 23:24:39.413516 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.413425    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29474ba6-d997-4d27-a609-7adc008c79ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.425732 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.425703    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46xl9\" (UniqueName: \"kubernetes.io/projected/76222ee2-ede8-4c5b-88ca-6da4400d27da-kube-api-access-46xl9\") pod \"aws-ebs-csi-driver-node-7nrfl\" (UID: \"76222ee2-ede8-4c5b-88ca-6da4400d27da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.425838 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.425794    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l95dz\" (UniqueName: \"kubernetes.io/projected/bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd-kube-api-access-l95dz\") pod \"iptables-alerter-ll5ls\" (UID: \"bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd\") " pod="openshift-network-operator/iptables-alerter-ll5ls"
Jun 03 23:24:39.427386 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.427364    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77w88\" (UniqueName: \"kubernetes.io/projected/29474ba6-d997-4d27-a609-7adc008c79ce-kube-api-access-77w88\") pod \"multus-additional-cni-plugins-rgddf\" (UID: \"29474ba6-d997-4d27-a609-7adc008c79ce\") " pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.427479 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.427465    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srbz6\" (UniqueName: \"kubernetes.io/projected/e42566fd-c3a2-48b7-8d03-a172d908aeff-kube-api-access-srbz6\") pod \"node-resolver-n99j6\" (UID: \"e42566fd-c3a2-48b7-8d03-a172d908aeff\") " pod="openshift-dns/node-resolver-n99j6"
Jun 03 23:24:39.506661 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.506589    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kx2mq"
Jun 03 23:24:39.506772 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.506717    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 03 23:24:39.513737 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.513714    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rkg7d"
Jun 03 23:24:39.518616 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:39.518592    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a1234d_160e_4994_834f_b87872953ed1.slice/crio-2e3957a5a06defa947593d7cf548754a039868bb119ae3abe3c14199730c4748 WatchSource:0}: Error finding container 2e3957a5a06defa947593d7cf548754a039868bb119ae3abe3c14199730c4748: Status 404 returned error can't find the container with id 2e3957a5a06defa947593d7cf548754a039868bb119ae3abe3c14199730c4748
Jun 03 23:24:39.522455 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.522421    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6n76l"
Jun 03 23:24:39.524808 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:39.524788    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e69283_7ec4_4312_8f28_dc75d920d672.slice/crio-36fb4d55211b4444378b917933734308440223a4c9a6efdcd965bc597dd2a859 WatchSource:0}: Error finding container 36fb4d55211b4444378b917933734308440223a4c9a6efdcd965bc597dd2a859: Status 404 returned error can't find the container with id 36fb4d55211b4444378b917933734308440223a4c9a6efdcd965bc597dd2a859
Jun 03 23:24:39.525745 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.525725    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bf9r5"
Jun 03 23:24:39.531017 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.530995    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:24:39.532345 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:39.532323    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb70ba3a5_03df_4847_9f88_12c8ae56099c.slice/crio-b174785b010de95ae3fa940c42a3a3bbb1f2c1db019f00944c511bd84afdc753 WatchSource:0}: Error finding container b174785b010de95ae3fa940c42a3a3bbb1f2c1db019f00944c511bd84afdc753: Status 404 returned error can't find the container with id b174785b010de95ae3fa940c42a3a3bbb1f2c1db019f00944c511bd84afdc753
Jun 03 23:24:39.535291 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:39.535268    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddff5f4ce_7ddd_4c84_9b4a_0170acb8c493.slice/crio-82f8e98c18abbdafdc1bd4cba440c4c00d4902249a24410fa48c1aa76d9543bb WatchSource:0}: Error finding container 82f8e98c18abbdafdc1bd4cba440c4c00d4902249a24410fa48c1aa76d9543bb: Status 404 returned error can't find the container with id 82f8e98c18abbdafdc1bd4cba440c4c00d4902249a24410fa48c1aa76d9543bb
Jun 03 23:24:39.536962 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.536785    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl"
Jun 03 23:24:39.540656 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.540627    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rgddf"
Jun 03 23:24:39.541034 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:39.540926    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60fa5a57_9316_4b22_a1ca_c5f40a4bc065.slice/crio-1d4f76f8492be5d349ed02977c06b2403a8de9eef51ce3215eb925e5b878c7bc WatchSource:0}: Error finding container 1d4f76f8492be5d349ed02977c06b2403a8de9eef51ce3215eb925e5b878c7bc: Status 404 returned error can't find the container with id 1d4f76f8492be5d349ed02977c06b2403a8de9eef51ce3215eb925e5b878c7bc
Jun 03 23:24:39.546930 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.546776    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n99j6"
Jun 03 23:24:39.547986 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:39.547956    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76222ee2_ede8_4c5b_88ca_6da4400d27da.slice/crio-30be0a853d6315608dd7efd21877d60e778b81f086aa02391304d0ad254959cf WatchSource:0}: Error finding container 30be0a853d6315608dd7efd21877d60e778b81f086aa02391304d0ad254959cf: Status 404 returned error can't find the container with id 30be0a853d6315608dd7efd21877d60e778b81f086aa02391304d0ad254959cf
Jun 03 23:24:39.551191 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.551059    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ll5ls"
Jun 03 23:24:39.552572 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:39.552542    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29474ba6_d997_4d27_a609_7adc008c79ce.slice/crio-df325fe562e43702d38e487d165aa5e7c65a6483d4fcc014b23cf3f48f940139 WatchSource:0}: Error finding container df325fe562e43702d38e487d165aa5e7c65a6483d4fcc014b23cf3f48f940139: Status 404 returned error can't find the container with id df325fe562e43702d38e487d165aa5e7c65a6483d4fcc014b23cf3f48f940139
Jun 03 23:24:39.558509 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:39.558481    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42566fd_c3a2_48b7_8d03_a172d908aeff.slice/crio-9e3843a0b3018c9c75c7ec2e285d152eb0f4893e8aace655b82047a49e3b56e0 WatchSource:0}: Error finding container 9e3843a0b3018c9c75c7ec2e285d152eb0f4893e8aace655b82047a49e3b56e0: Status 404 returned error can't find the container with id 9e3843a0b3018c9c75c7ec2e285d152eb0f4893e8aace655b82047a49e3b56e0
Jun 03 23:24:39.561385 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:24:39.561356    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd1e1a9b_a778_4d53_95a4_3b4c3adf6efd.slice/crio-342805c02a21df4578d5c4cc4ba9248899215e2c6c323a3a39a19bc4dfca1ada WatchSource:0}: Error finding container 342805c02a21df4578d5c4cc4ba9248899215e2c6c323a3a39a19bc4dfca1ada: Status 404 returned error can't find the container with id 342805c02a21df4578d5c4cc4ba9248899215e2c6c323a3a39a19bc4dfca1ada
Jun 03 23:24:39.815709 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.815676    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs\") pod \"network-metrics-daemon-ssbpc\" (UID: \"81ee1ff9-1018-4eaf-b246-18b5f7a95f78\") " pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:39.815889 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.815861    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 23:24:39.815956 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.815918    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs podName:81ee1ff9-1018-4eaf-b246-18b5f7a95f78 nodeName:}" failed. No retries permitted until 2026-06-03 23:24:40.815899709 +0000 UTC m=+2.978577762 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs") pod "network-metrics-daemon-ssbpc" (UID: "81ee1ff9-1018-4eaf-b246-18b5f7a95f78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 23:24:39.917312 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.916501    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dggl\" (UniqueName: \"kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl\") pod \"network-check-target-9gpsw\" (UID: \"7ab2d105-78a7-4089-9f3b-f0b03411ec50\") " pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:39.917312 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:39.916587    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret\") pod \"global-pull-secret-syncer-9d2cb\" (UID: \"55ceda9f-6200-4a21-a050-d290d631043f\") " pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:39.917312 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.916717    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 03 23:24:39.917312 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.916782    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret podName:55ceda9f-6200-4a21-a050-d290d631043f nodeName:}" failed. No retries permitted until 2026-06-03 23:24:40.916755005 +0000 UTC m=+3.079433074 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret") pod "global-pull-secret-syncer-9d2cb" (UID: "55ceda9f-6200-4a21-a050-d290d631043f") : object "kube-system"/"original-pull-secret" not registered
Jun 03 23:24:39.917312 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.917182    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 03 23:24:39.917312 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.917201    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 03 23:24:39.917312 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.917230    2577 projected.go:196] Error preparing data for projected volume kube-api-access-6dggl for pod openshift-network-diagnostics/network-check-target-9gpsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 23:24:39.917312 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:39.917272    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl podName:7ab2d105-78a7-4089-9f3b-f0b03411ec50 nodeName:}" failed. No retries permitted until 2026-06-03 23:24:40.917257765 +0000 UTC m=+3.079935825 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dggl" (UniqueName: "kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl") pod "network-check-target-9gpsw" (UID: "7ab2d105-78a7-4089-9f3b-f0b03411ec50") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 23:24:40.053038 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:40.053000    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 03 23:24:40.246771 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:40.246638    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 23:19:39 +0000 UTC" deadline="2027-11-19 19:59:58.288854034 +0000 UTC"
Jun 03 23:24:40.246771 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:40.246696    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12812h35m18.042163425s"
Jun 03 23:24:40.300193 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:40.300104    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n99j6" event={"ID":"e42566fd-c3a2-48b7-8d03-a172d908aeff","Type":"ContainerStarted","Data":"9e3843a0b3018c9c75c7ec2e285d152eb0f4893e8aace655b82047a49e3b56e0"}
Jun 03 23:24:40.316323 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:40.316286    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rgddf" event={"ID":"29474ba6-d997-4d27-a609-7adc008c79ce","Type":"ContainerStarted","Data":"df325fe562e43702d38e487d165aa5e7c65a6483d4fcc014b23cf3f48f940139"}
Jun 03 23:24:40.321262 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:40.321230    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl" event={"ID":"76222ee2-ede8-4c5b-88ca-6da4400d27da","Type":"ContainerStarted","Data":"30be0a853d6315608dd7efd21877d60e778b81f086aa02391304d0ad254959cf"}
Jun 03 23:24:40.336860 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:40.336823    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bf9r5" event={"ID":"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493","Type":"ContainerStarted","Data":"82f8e98c18abbdafdc1bd4cba440c4c00d4902249a24410fa48c1aa76d9543bb"}
Jun 03 23:24:40.374378 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:40.374221    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rkg7d" event={"ID":"06e69283-7ec4-4312-8f28-dc75d920d672","Type":"ContainerStarted","Data":"36fb4d55211b4444378b917933734308440223a4c9a6efdcd965bc597dd2a859"}
Jun 03 23:24:40.377018 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:40.376978    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kx2mq" event={"ID":"f0a1234d-160e-4994-834f-b87872953ed1","Type":"ContainerStarted","Data":"2e3957a5a06defa947593d7cf548754a039868bb119ae3abe3c14199730c4748"}
Jun 03 23:24:40.381018 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:40.380985    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ll5ls" event={"ID":"bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd","Type":"ContainerStarted","Data":"342805c02a21df4578d5c4cc4ba9248899215e2c6c323a3a39a19bc4dfca1ada"}
Jun 03 23:24:40.388416 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:40.388327    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr" event={"ID":"60fa5a57-9316-4b22-a1ca-c5f40a4bc065","Type":"ContainerStarted","Data":"1d4f76f8492be5d349ed02977c06b2403a8de9eef51ce3215eb925e5b878c7bc"}
Jun 03 23:24:40.394445 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:40.394419    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6n76l" event={"ID":"b70ba3a5-03df-4847-9f88-12c8ae56099c","Type":"ContainerStarted","Data":"b174785b010de95ae3fa940c42a3a3bbb1f2c1db019f00944c511bd84afdc753"}
Jun 03 23:24:40.828317 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:40.828121    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs\") pod \"network-metrics-daemon-ssbpc\" (UID: \"81ee1ff9-1018-4eaf-b246-18b5f7a95f78\") " pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:40.828317 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:40.828317    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 23:24:40.828606 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:40.828383    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs podName:81ee1ff9-1018-4eaf-b246-18b5f7a95f78 nodeName:}" failed. No retries permitted until 2026-06-03 23:24:42.828362871 +0000 UTC m=+4.991040928 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs") pod "network-metrics-daemon-ssbpc" (UID: "81ee1ff9-1018-4eaf-b246-18b5f7a95f78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 23:24:40.884938 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:40.884912    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 03 23:24:40.929188 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:40.929152    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dggl\" (UniqueName: \"kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl\") pod \"network-check-target-9gpsw\" (UID: \"7ab2d105-78a7-4089-9f3b-f0b03411ec50\") " pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:40.929373 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:40.929240    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret\") pod \"global-pull-secret-syncer-9d2cb\" (UID: \"55ceda9f-6200-4a21-a050-d290d631043f\") " pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:40.929373 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:40.929357    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 03 23:24:40.929815 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:40.929485    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret podName:55ceda9f-6200-4a21-a050-d290d631043f nodeName:}" failed. No retries permitted until 2026-06-03 23:24:42.929461505 +0000 UTC m=+5.092139560 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret") pod "global-pull-secret-syncer-9d2cb" (UID: "55ceda9f-6200-4a21-a050-d290d631043f") : object "kube-system"/"original-pull-secret" not registered
Jun 03 23:24:40.929815 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:40.929612    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 03 23:24:40.929815 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:40.929632    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 03 23:24:40.929815 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:40.929644    2577 projected.go:196] Error preparing data for projected volume kube-api-access-6dggl for pod openshift-network-diagnostics/network-check-target-9gpsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 23:24:40.929815 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:40.929685    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl podName:7ab2d105-78a7-4089-9f3b-f0b03411ec50 nodeName:}" failed. No retries permitted until 2026-06-03 23:24:42.929672668 +0000 UTC m=+5.092350736 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dggl" (UniqueName: "kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl") pod "network-check-target-9gpsw" (UID: "7ab2d105-78a7-4089-9f3b-f0b03411ec50") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 23:24:41.247418 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:41.247269    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 23:19:39 +0000 UTC" deadline="2028-01-15 07:46:57.196611043 +0000 UTC"
Jun 03 23:24:41.247418 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:41.247333    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14168h22m15.949281639s"
Jun 03 23:24:41.272873 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:41.272847    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:41.273034 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:41.272977    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:24:41.273453 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:41.273430    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:41.273565 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:41.273528    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:24:41.273635 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:41.273599    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:41.273687 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:41.273667    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:24:42.846352 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:42.845766    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs\") pod \"network-metrics-daemon-ssbpc\" (UID: \"81ee1ff9-1018-4eaf-b246-18b5f7a95f78\") " pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:42.846352 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:42.845921    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 23:24:42.846352 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:42.845987    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs podName:81ee1ff9-1018-4eaf-b246-18b5f7a95f78 nodeName:}" failed. No retries permitted until 2026-06-03 23:24:46.845968103 +0000 UTC m=+9.008646155 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs") pod "network-metrics-daemon-ssbpc" (UID: "81ee1ff9-1018-4eaf-b246-18b5f7a95f78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 23:24:42.946574 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:42.946415    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret\") pod \"global-pull-secret-syncer-9d2cb\" (UID: \"55ceda9f-6200-4a21-a050-d290d631043f\") " pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:42.946574 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:42.946497    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dggl\" (UniqueName: \"kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl\") pod \"network-check-target-9gpsw\" (UID: \"7ab2d105-78a7-4089-9f3b-f0b03411ec50\") " pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:42.946574 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:42.946545    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 03 23:24:42.946885 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:42.946626    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret podName:55ceda9f-6200-4a21-a050-d290d631043f nodeName:}" failed. No retries permitted until 2026-06-03 23:24:46.946606314 +0000 UTC m=+9.109284372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret") pod "global-pull-secret-syncer-9d2cb" (UID: "55ceda9f-6200-4a21-a050-d290d631043f") : object "kube-system"/"original-pull-secret" not registered
Jun 03 23:24:42.946885 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:42.946638    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 03 23:24:42.946885 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:42.946653    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 03 23:24:42.946885 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:42.946666    2577 projected.go:196] Error preparing data for projected volume kube-api-access-6dggl for pod openshift-network-diagnostics/network-check-target-9gpsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 23:24:42.946885 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:42.946714    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl podName:7ab2d105-78a7-4089-9f3b-f0b03411ec50 nodeName:}" failed. No retries permitted until 2026-06-03 23:24:46.946698275 +0000 UTC m=+9.109376330 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dggl" (UniqueName: "kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl") pod "network-check-target-9gpsw" (UID: "7ab2d105-78a7-4089-9f3b-f0b03411ec50") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 23:24:43.273130 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:43.273057    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:43.273309 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:43.273217    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:24:43.273309 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:43.273294    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:43.273438 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:43.273344    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:43.273438 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:43.273415    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:24:43.273538 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:43.273510    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:24:45.273046 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:45.272964    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:45.273046 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:45.273006    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:45.273586 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:45.273094    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:45.273586 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:45.273113    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:24:45.273586 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:45.273443    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:24:45.273586 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:45.273545    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:24:46.880443 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:46.880407    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs\") pod \"network-metrics-daemon-ssbpc\" (UID: \"81ee1ff9-1018-4eaf-b246-18b5f7a95f78\") " pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:46.880905 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:46.880579    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 23:24:46.880905 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:46.880645    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs podName:81ee1ff9-1018-4eaf-b246-18b5f7a95f78 nodeName:}" failed. No retries permitted until 2026-06-03 23:24:54.880628796 +0000 UTC m=+17.043306851 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs") pod "network-metrics-daemon-ssbpc" (UID: "81ee1ff9-1018-4eaf-b246-18b5f7a95f78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 23:24:46.981389 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:46.981326    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dggl\" (UniqueName: \"kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl\") pod \"network-check-target-9gpsw\" (UID: \"7ab2d105-78a7-4089-9f3b-f0b03411ec50\") " pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:46.981610 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:46.981400    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret\") pod \"global-pull-secret-syncer-9d2cb\" (UID: \"55ceda9f-6200-4a21-a050-d290d631043f\") " pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:46.981610 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:46.981521    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 03 23:24:46.981610 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:46.981580    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret podName:55ceda9f-6200-4a21-a050-d290d631043f nodeName:}" failed. No retries permitted until 2026-06-03 23:24:54.981563081 +0000 UTC m=+17.144241149 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret") pod "global-pull-secret-syncer-9d2cb" (UID: "55ceda9f-6200-4a21-a050-d290d631043f") : object "kube-system"/"original-pull-secret" not registered
Jun 03 23:24:46.981951 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:46.981923    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 03 23:24:46.981951 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:46.981953    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 03 23:24:46.982130 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:46.981967    2577 projected.go:196] Error preparing data for projected volume kube-api-access-6dggl for pod openshift-network-diagnostics/network-check-target-9gpsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 23:24:46.982130 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:46.982048    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl podName:7ab2d105-78a7-4089-9f3b-f0b03411ec50 nodeName:}" failed. No retries permitted until 2026-06-03 23:24:54.982032735 +0000 UTC m=+17.144710803 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dggl" (UniqueName: "kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl") pod "network-check-target-9gpsw" (UID: "7ab2d105-78a7-4089-9f3b-f0b03411ec50") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 23:24:47.272927 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:47.272830    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:47.273088 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:47.272954    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:24:47.273466 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:47.273271    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:47.273466 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:47.273285    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:47.273466 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:47.273413    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:24:47.273685 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:47.273502    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:24:49.273407 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:49.273379    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:49.273845 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:49.273382    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:49.273845 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:49.273389    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:49.273845 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:49.273585    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:24:49.273845 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:49.273634    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:24:49.273845 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:49.273479    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:24:51.273391 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:51.273353    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:51.273807 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:51.273393    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:51.273807 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:51.273395    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:51.273807 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:51.273486    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:24:51.273807 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:51.273583    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:24:51.273807 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:51.273667    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:24:53.273359 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:53.273329    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:53.273868 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:53.273438    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:53.273868 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:53.273451    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:24:53.273868 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:53.273468    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:53.273868 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:53.273552    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:24:53.273868 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:53.273660    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:24:54.939395 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:54.939356    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs\") pod \"network-metrics-daemon-ssbpc\" (UID: \"81ee1ff9-1018-4eaf-b246-18b5f7a95f78\") " pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:54.939862 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:54.939518    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 23:24:54.939862 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:54.939603    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs podName:81ee1ff9-1018-4eaf-b246-18b5f7a95f78 nodeName:}" failed. No retries permitted until 2026-06-03 23:25:10.93958554 +0000 UTC m=+33.102263593 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs") pod "network-metrics-daemon-ssbpc" (UID: "81ee1ff9-1018-4eaf-b246-18b5f7a95f78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 23:24:55.040065 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:55.040028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dggl\" (UniqueName: \"kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl\") pod \"network-check-target-9gpsw\" (UID: \"7ab2d105-78a7-4089-9f3b-f0b03411ec50\") " pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:55.040272 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:55.040110    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret\") pod \"global-pull-secret-syncer-9d2cb\" (UID: \"55ceda9f-6200-4a21-a050-d290d631043f\") " pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:55.040272 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:55.040235    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 03 23:24:55.040272 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:55.040260    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 03 23:24:55.040272 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:55.040271    2577 projected.go:196] Error preparing data for projected volume kube-api-access-6dggl for pod openshift-network-diagnostics/network-check-target-9gpsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 23:24:55.040438 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:55.040322    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl podName:7ab2d105-78a7-4089-9f3b-f0b03411ec50 nodeName:}" failed. No retries permitted until 2026-06-03 23:25:11.040305762 +0000 UTC m=+33.202983815 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dggl" (UniqueName: "kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl") pod "network-check-target-9gpsw" (UID: "7ab2d105-78a7-4089-9f3b-f0b03411ec50") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 23:24:55.040438 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:55.040238    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 03 23:24:55.040438 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:55.040385    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret podName:55ceda9f-6200-4a21-a050-d290d631043f nodeName:}" failed. No retries permitted until 2026-06-03 23:25:11.040369383 +0000 UTC m=+33.203047440 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret") pod "global-pull-secret-syncer-9d2cb" (UID: "55ceda9f-6200-4a21-a050-d290d631043f") : object "kube-system"/"original-pull-secret" not registered
Jun 03 23:24:55.273098 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:55.273006    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:55.273098 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:55.273037    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:55.273098 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:55.273026    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:55.273404 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:55.273138    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:24:55.273404 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:55.273259    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:24:55.273404 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:55.273352    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:24:57.272911 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:57.272876    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:57.273393 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:57.272997    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:57.273393 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:57.273022    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:57.273393 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:57.273014    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:24:57.273393 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:57.273102    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:24:57.273393 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:57.273166    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:24:59.273718 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:59.273436    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:24:59.274352 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:59.274224    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:24:59.274352 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:59.273903    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:24:59.274487 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:59.274388    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:24:59.274487 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:59.274458    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:24:59.274588 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:24:59.274562    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:24:59.437800 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:59.437699    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-110.ec2.internal" event={"ID":"a14572b805eb35459a9a5413dc15e576","Type":"ContainerStarted","Data":"1229a72eafd9822a5232d607eb549f155a7e0513656c536459d626bb26b98bdd"}
Jun 03 23:24:59.443139 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:59.442762    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr" event={"ID":"60fa5a57-9316-4b22-a1ca-c5f40a4bc065","Type":"ContainerStarted","Data":"5dd70020af9d3fef53727260b7302070ef932aaed5c351e8098a36eab339c6d0"}
Jun 03 23:24:59.443139 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:59.442844    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr" event={"ID":"60fa5a57-9316-4b22-a1ca-c5f40a4bc065","Type":"ContainerStarted","Data":"dc2bf477f3f3352326e2ab5d9452fcdea51e071500164050cf71c2181233ccbb"}
Jun 03 23:24:59.443139 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:59.442892    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr" event={"ID":"60fa5a57-9316-4b22-a1ca-c5f40a4bc065","Type":"ContainerStarted","Data":"23e6669fa1a2f908eccd53f6eaa51413cdfeb15e112a8d9f6d7a208aa69ea54a"}
Jun 03 23:24:59.443139 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:59.442945    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr" event={"ID":"60fa5a57-9316-4b22-a1ca-c5f40a4bc065","Type":"ContainerStarted","Data":"c807792de3dd3364847b170e2e1284862bf5518bfa09e997cbaf5ab89528459d"}
Jun 03 23:24:59.443139 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:59.442993    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr" event={"ID":"60fa5a57-9316-4b22-a1ca-c5f40a4bc065","Type":"ContainerStarted","Data":"34adea2700725065d5e0128fe2a5e1317deb143a229132f0679c20d8f5ea3fb1"}
Jun 03 23:24:59.445124 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:59.445099    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bf9r5" event={"ID":"dff5f4ce-7ddd-4c84-9b4a-0170acb8c493","Type":"ContainerStarted","Data":"9dd99a744a5cbc866c96d5d8e4efa55a40e747960bf03edcfce15b442ec14479"}
Jun 03 23:24:59.446335 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:59.446314    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rkg7d" event={"ID":"06e69283-7ec4-4312-8f28-dc75d920d672","Type":"ContainerStarted","Data":"e945d13c174590e400e32983bc74713bdbd10ea40bd7b306839bb86a77149664"}
Jun 03 23:24:59.455788 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:59.455698    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-110.ec2.internal" podStartSLOduration=20.455680798 podStartE2EDuration="20.455680798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-03 23:24:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:24:59.455157886 +0000 UTC m=+21.617835962" watchObservedRunningTime="2026-06-03 23:24:59.455680798 +0000 UTC m=+21.618358873"
Jun 03 23:24:59.474471 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:59.474262    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rkg7d" podStartSLOduration=2.538913241 podStartE2EDuration="21.474243359s" totalImagesPullingTime="18.935330118s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-03 23:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:24:59.473598609 +0000 UTC m=+21.636276683" watchObservedRunningTime="2026-06-03 23:24:59.474243359 +0000 UTC m=+21.636921429"
Jun 03 23:24:59.493075 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:24:59.493024    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bf9r5" podStartSLOduration=2.580452244 podStartE2EDuration="21.49300573s" totalImagesPullingTime="18.912553486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-03 23:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:24:59.492710006 +0000 UTC m=+21.655388081" watchObservedRunningTime="2026-06-03 23:24:59.49300573 +0000 UTC m=+21.655683806"
Jun 03 23:25:00.449422 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:00.449386    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ll5ls" event={"ID":"bd1e1a9b-a778-4d53-95a4-3b4c3adf6efd","Type":"ContainerStarted","Data":"0e852a3771f9af3aa6b452dbf563b165f0006607e62652cc8f06c7b5c2e16190"}
Jun 03 23:25:00.452054 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:00.452030    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr" event={"ID":"60fa5a57-9316-4b22-a1ca-c5f40a4bc065","Type":"ContainerStarted","Data":"831d21dbd3840514d4f67461babc0b8cbb9cd6a3ed42f223a5c565d6e0d98458"}
Jun 03 23:25:00.453493 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:00.453464    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6n76l" event={"ID":"b70ba3a5-03df-4847-9f88-12c8ae56099c","Type":"ContainerStarted","Data":"e6bcabe6e95380a4b6accbb9f0b52102ebbdb791009b0e02ec7014bb1655e5fc"}
Jun 03 23:25:00.454979 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:00.454955    2577 generic.go:356] "Generic (PLEG): container finished" podID="da2bd6e93c11d61b803364c8204bbd7a" containerID="ab3c9cff4574e52fe4b9c4a0460f41dfbd08ff840fa7cc11bafd00d9ccc52006" exitCode=0
Jun 03 23:25:00.455076 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:00.455018    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal" event={"ID":"da2bd6e93c11d61b803364c8204bbd7a","Type":"ContainerDied","Data":"ab3c9cff4574e52fe4b9c4a0460f41dfbd08ff840fa7cc11bafd00d9ccc52006"}
Jun 03 23:25:00.456348 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:00.456329    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n99j6" event={"ID":"e42566fd-c3a2-48b7-8d03-a172d908aeff","Type":"ContainerStarted","Data":"9fb8344fa529c9c0954cab317e774c6601fe4f42a2bf473ff68482a2f6e0dd4a"}
Jun 03 23:25:00.457717 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:00.457688    2577 generic.go:356] "Generic (PLEG): container finished" podID="29474ba6-d997-4d27-a609-7adc008c79ce" containerID="6700e35099b052ede85944d08dbf482df083fea7342952b103979090064c9f0b" exitCode=0
Jun 03 23:25:00.457786 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:00.457769    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rgddf" event={"ID":"29474ba6-d997-4d27-a609-7adc008c79ce","Type":"ContainerDied","Data":"6700e35099b052ede85944d08dbf482df083fea7342952b103979090064c9f0b"}
Jun 03 23:25:00.459064 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:00.459047    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl" event={"ID":"76222ee2-ede8-4c5b-88ca-6da4400d27da","Type":"ContainerStarted","Data":"70fd57fcabefc32acce0582b0f520f4069a10321d301f17fe99c8a3e8663138b"}
Jun 03 23:25:00.460278 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:00.460242    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kx2mq" event={"ID":"f0a1234d-160e-4994-834f-b87872953ed1","Type":"ContainerStarted","Data":"dcdfc70e38932ca7ac677513ec5097c85c611ba0f7ec003038fa13722bf9b909"}
Jun 03 23:25:00.464531 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:00.464491    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ll5ls" podStartSLOduration=3.5749211929999998 podStartE2EDuration="22.464478237s" totalImagesPullingTime="18.889557044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-03 23:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:25:00.464349616 +0000 UTC m=+22.627027691" watchObservedRunningTime="2026-06-03 23:25:00.464478237 +0000 UTC m=+22.627156314"
Jun 03 23:25:00.477899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:00.477860    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-n99j6" podStartSLOduration=3.592353209 podStartE2EDuration="22.477849358s" totalImagesPullingTime="18.885496149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-03 23:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:25:00.477726404 +0000 UTC m=+22.640404512" watchObservedRunningTime="2026-06-03 23:25:00.477849358 +0000 UTC m=+22.640527432"
Jun 03 23:25:00.495146 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:00.495109    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6n76l" podStartSLOduration=3.60109033 podStartE2EDuration="22.495098844s" totalImagesPullingTime="18.894008514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-03 23:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:25:00.494985186 +0000 UTC m=+22.657663271" watchObservedRunningTime="2026-06-03 23:25:00.495098844 +0000 UTC m=+22.657776928"
Jun 03 23:25:00.550982 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:00.550949    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kx2mq" podStartSLOduration=3.643348351 podStartE2EDuration="22.550939706s" totalImagesPullingTime="18.907591355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:25:00.550765715 +0000 UTC m=+22.713443790" watchObservedRunningTime="2026-06-03 23:25:00.550939706 +0000 UTC m=+22.713617791"
Jun 03 23:25:00.697074 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:00.696937    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jun 03 23:25:01.263316 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:01.263181    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-03T23:25:00.696956651Z","UUID":"32061561-23d6-4bb5-b330-d212a7da7371","Handler":null,"Name":"","Endpoint":""}
Jun 03 23:25:01.265129 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:01.265104    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jun 03 23:25:01.265288 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:01.265152    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jun 03 23:25:01.272624 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:01.272567    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:25:01.272624 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:01.272591    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:25:01.272778 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:01.272698    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:25:01.272778 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:01.272744    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:25:01.272860 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:01.272811    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:25:01.273028 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:01.272905    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:25:01.465854 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:01.465810    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal" event={"ID":"da2bd6e93c11d61b803364c8204bbd7a","Type":"ContainerStarted","Data":"2a57f2f15098c0ee7d569862a3e5d4c9419fd2b837ef5981fcdd888c67ed76a0"}
Jun 03 23:25:01.468179 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:01.468151    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl" event={"ID":"76222ee2-ede8-4c5b-88ca-6da4400d27da","Type":"ContainerStarted","Data":"55784c30834b31aff536d8ac1c2742e3eb399c076943866f2757cd9c00ce43d4"}
Jun 03 23:25:01.484906 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:01.484849    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-110.ec2.internal" podStartSLOduration=22.484829149 podStartE2EDuration="22.484829149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-03 23:24:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:25:01.484103207 +0000 UTC m=+23.646781280" watchObservedRunningTime="2026-06-03 23:25:01.484829149 +0000 UTC m=+23.647507224"
Jun 03 23:25:02.474154 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:02.473803    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr" event={"ID":"60fa5a57-9316-4b22-a1ca-c5f40a4bc065","Type":"ContainerStarted","Data":"c4644c6daea4c417c4c7f7d9bc2becf2209978032671609703cf465804bd2508"}
Jun 03 23:25:02.476115 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:02.476087    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl" event={"ID":"76222ee2-ede8-4c5b-88ca-6da4400d27da","Type":"ContainerStarted","Data":"cd514081790537c24535933de08a4860182dc99217803fc9f431643ed7068e4b"}
Jun 03 23:25:02.494846 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:02.494803    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7nrfl" podStartSLOduration=3.32861104 podStartE2EDuration="24.494790434s" totalImagesPullingTime="21.166179394s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-03 23:24:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:25:02.494345259 +0000 UTC m=+24.657023333" watchObservedRunningTime="2026-06-03 23:25:02.494790434 +0000 UTC m=+24.657468521"
Jun 03 23:25:03.273369 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:03.273335    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:25:03.273549 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:03.273342    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:25:03.273549 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:03.273458    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:25:03.273549 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:03.273342    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:25:03.273549 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:03.273526    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:25:03.273742 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:03.273592    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:25:03.575316 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:03.575286    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kx2mq"
Jun 03 23:25:03.575991 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:03.575969    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kx2mq"
Jun 03 23:25:05.273259 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:05.273029    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:25:05.273606 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:05.273078    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:25:05.273606 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:05.273308    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:25:05.273606 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:05.273103    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:25:05.273606 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:05.273372    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:25:05.273606 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:05.273447    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:25:05.484509 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:05.484472    2577 generic.go:356] "Generic (PLEG): container finished" podID="29474ba6-d997-4d27-a609-7adc008c79ce" containerID="57329a544b9f89aff25591fb2ee28e83950db0ee004dea75c7c21979ffa232b8" exitCode=0
Jun 03 23:25:05.484692 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:05.484553    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rgddf" event={"ID":"29474ba6-d997-4d27-a609-7adc008c79ce","Type":"ContainerDied","Data":"57329a544b9f89aff25591fb2ee28e83950db0ee004dea75c7c21979ffa232b8"}
Jun 03 23:25:05.488390 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:05.488367    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr" event={"ID":"60fa5a57-9316-4b22-a1ca-c5f40a4bc065","Type":"ContainerStarted","Data":"acad9eb9f4c61e7cd2f98df145af8e76254568248941db23157df58152e7e0de"}
Jun 03 23:25:05.488709 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:05.488692    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:25:05.488819 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:05.488807    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:25:05.488877 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:05.488859    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:25:05.503803 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:05.503786    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:25:05.503989 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:05.503975    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:25:05.534896 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:05.534826    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr" podStartSLOduration=8.143889032 podStartE2EDuration="27.534813771s" totalImagesPullingTime="19.390924739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-03 23:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:25:05.534643992 +0000 UTC m=+27.697322065" watchObservedRunningTime="2026-06-03 23:25:05.534813771 +0000 UTC m=+27.697491855"
Jun 03 23:25:06.493788 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:06.493760    2577 generic.go:356] "Generic (PLEG): container finished" podID="29474ba6-d997-4d27-a609-7adc008c79ce" containerID="ad1774d30c006b947653213f91f41709718033ebeeb7ca8976f4aded461ba628" exitCode=0
Jun 03 23:25:06.495000 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:06.493853    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rgddf" event={"ID":"29474ba6-d997-4d27-a609-7adc008c79ce","Type":"ContainerDied","Data":"ad1774d30c006b947653213f91f41709718033ebeeb7ca8976f4aded461ba628"}
Jun 03 23:25:06.652924 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:06.652894    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9gpsw"]
Jun 03 23:25:06.653085 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:06.652993    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:25:06.653151 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:06.653081    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:25:06.653539 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:06.653509    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9d2cb"]
Jun 03 23:25:06.653655 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:06.653585    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:25:06.653704 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:06.653669    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:25:06.655476 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:06.655456    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ssbpc"]
Jun 03 23:25:06.655556 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:06.655527    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:25:06.655617 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:06.655600    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:25:07.172998 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:07.172961    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kx2mq"
Jun 03 23:25:07.173177 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:07.173121    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jun 03 23:25:07.173569 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:07.173554    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kx2mq"
Jun 03 23:25:08.273774 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:08.273519    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:25:08.274225 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:08.273592    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:25:08.274225 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:08.273783    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:25:08.274225 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:08.273897    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:25:08.274225 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:08.273625    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:25:08.274225 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:08.273987    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:25:08.500272 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:08.500228    2577 generic.go:356] "Generic (PLEG): container finished" podID="29474ba6-d997-4d27-a609-7adc008c79ce" containerID="cb2f63fe282b37566f3874909367f78825d329c9ebe6b0b488924076118403f6" exitCode=0
Jun 03 23:25:08.500433 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:08.500281    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rgddf" event={"ID":"29474ba6-d997-4d27-a609-7adc008c79ce","Type":"ContainerDied","Data":"cb2f63fe282b37566f3874909367f78825d329c9ebe6b0b488924076118403f6"}
Jun 03 23:25:10.272899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:10.272773    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:25:10.272899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:10.272784    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:25:10.273461 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:10.272897    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gpsw" podUID="7ab2d105-78a7-4089-9f3b-f0b03411ec50"
Jun 03 23:25:10.273461 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:10.272995    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:25:10.273461 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:10.273060    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:25:10.273461 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:10.273134    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9d2cb" podUID="55ceda9f-6200-4a21-a050-d290d631043f"
Jun 03 23:25:10.968562 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:10.968528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs\") pod \"network-metrics-daemon-ssbpc\" (UID: \"81ee1ff9-1018-4eaf-b246-18b5f7a95f78\") " pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:25:10.968714 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:10.968636    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 23:25:10.968714 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:10.968703    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs podName:81ee1ff9-1018-4eaf-b246-18b5f7a95f78 nodeName:}" failed. No retries permitted until 2026-06-03 23:25:42.968683517 +0000 UTC m=+65.131361574 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs") pod "network-metrics-daemon-ssbpc" (UID: "81ee1ff9-1018-4eaf-b246-18b5f7a95f78") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 23:25:11.068901 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.068865    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dggl\" (UniqueName: \"kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl\") pod \"network-check-target-9gpsw\" (UID: \"7ab2d105-78a7-4089-9f3b-f0b03411ec50\") " pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:25:11.069076 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.068945    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret\") pod \"global-pull-secret-syncer-9d2cb\" (UID: \"55ceda9f-6200-4a21-a050-d290d631043f\") " pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:25:11.069076 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:11.069025    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 03 23:25:11.069188 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:11.069090    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret podName:55ceda9f-6200-4a21-a050-d290d631043f nodeName:}" failed. No retries permitted until 2026-06-03 23:25:43.069074823 +0000 UTC m=+65.231752875 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret") pod "global-pull-secret-syncer-9d2cb" (UID: "55ceda9f-6200-4a21-a050-d290d631043f") : object "kube-system"/"original-pull-secret" not registered
Jun 03 23:25:11.069188 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:11.069025    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 03 23:25:11.069188 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:11.069139    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 03 23:25:11.069188 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:11.069152    2577 projected.go:196] Error preparing data for projected volume kube-api-access-6dggl for pod openshift-network-diagnostics/network-check-target-9gpsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 23:25:11.069359 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:11.069198    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl podName:7ab2d105-78a7-4089-9f3b-f0b03411ec50 nodeName:}" failed. No retries permitted until 2026-06-03 23:25:43.069185775 +0000 UTC m=+65.231863840 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6dggl" (UniqueName: "kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl") pod "network-check-target-9gpsw" (UID: "7ab2d105-78a7-4089-9f3b-f0b03411ec50") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 23:25:11.649358 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.649328    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-110.ec2.internal" event="NodeReady"
Jun 03 23:25:11.649863 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.649490    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jun 03 23:25:11.697216 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.697184    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5q42g"]
Jun 03 23:25:11.700256 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.700224    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-28f9z"]
Jun 03 23:25:11.700404 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.700313    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:11.701963 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.701943    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-28f9z"
Jun 03 23:25:11.702512 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.702490    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jun 03 23:25:11.702623 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.702495    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6h75n\""
Jun 03 23:25:11.703009 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.702994    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jun 03 23:25:11.704319 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.704274    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jun 03 23:25:11.704440 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.704342    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jun 03 23:25:11.704548 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.704511    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-p5tlk\""
Jun 03 23:25:11.704621 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.704549    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jun 03 23:25:11.711540 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.711518    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5q42g"]
Jun 03 23:25:11.713965 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.713943    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-28f9z"]
Jun 03 23:25:11.875603 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.875569    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a1a6edf-938e-4784-bbc6-74c8829977a2-config-volume\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:11.875603 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.875607    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a1a6edf-938e-4784-bbc6-74c8829977a2-tmp-dir\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:11.875830 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.875633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdwqg\" (UniqueName: \"kubernetes.io/projected/3a1a6edf-938e-4784-bbc6-74c8829977a2-kube-api-access-rdwqg\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:11.875830 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.875707    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sccld\" (UniqueName: \"kubernetes.io/projected/201466a6-58e5-49a6-869f-2161d2c8b708-kube-api-access-sccld\") pod \"ingress-canary-28f9z\" (UID: \"201466a6-58e5-49a6-869f-2161d2c8b708\") " pod="openshift-ingress-canary/ingress-canary-28f9z"
Jun 03 23:25:11.875830 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.875750    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/201466a6-58e5-49a6-869f-2161d2c8b708-cert\") pod \"ingress-canary-28f9z\" (UID: \"201466a6-58e5-49a6-869f-2161d2c8b708\") " pod="openshift-ingress-canary/ingress-canary-28f9z"
Jun 03 23:25:11.875830 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.875808    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:11.976516 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.976427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a1a6edf-938e-4784-bbc6-74c8829977a2-config-volume\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:11.976516 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.976499    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a1a6edf-938e-4784-bbc6-74c8829977a2-tmp-dir\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:11.976723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.976536    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdwqg\" (UniqueName: \"kubernetes.io/projected/3a1a6edf-938e-4784-bbc6-74c8829977a2-kube-api-access-rdwqg\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:11.976723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.976563    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sccld\" (UniqueName: \"kubernetes.io/projected/201466a6-58e5-49a6-869f-2161d2c8b708-kube-api-access-sccld\") pod \"ingress-canary-28f9z\" (UID: \"201466a6-58e5-49a6-869f-2161d2c8b708\") " pod="openshift-ingress-canary/ingress-canary-28f9z"
Jun 03 23:25:11.976723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.976593    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/201466a6-58e5-49a6-869f-2161d2c8b708-cert\") pod \"ingress-canary-28f9z\" (UID: \"201466a6-58e5-49a6-869f-2161d2c8b708\") " pod="openshift-ingress-canary/ingress-canary-28f9z"
Jun 03 23:25:11.976723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.976665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:11.976918 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:11.976792    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 23:25:11.976918 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:11.976845    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls podName:3a1a6edf-938e-4784-bbc6-74c8829977a2 nodeName:}" failed. No retries permitted until 2026-06-03 23:25:12.4768286 +0000 UTC m=+34.639506652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls") pod "dns-default-5q42g" (UID: "3a1a6edf-938e-4784-bbc6-74c8829977a2") : secret "dns-default-metrics-tls" not found
Jun 03 23:25:11.976918 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.976901    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a1a6edf-938e-4784-bbc6-74c8829977a2-tmp-dir\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:11.977093 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.977065    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a1a6edf-938e-4784-bbc6-74c8829977a2-config-volume\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:11.982236 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.982199    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/201466a6-58e5-49a6-869f-2161d2c8b708-cert\") pod \"ingress-canary-28f9z\" (UID: \"201466a6-58e5-49a6-869f-2161d2c8b708\") " pod="openshift-ingress-canary/ingress-canary-28f9z"
Jun 03 23:25:11.987390 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.986423    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdwqg\" (UniqueName: \"kubernetes.io/projected/3a1a6edf-938e-4784-bbc6-74c8829977a2-kube-api-access-rdwqg\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:11.988552 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:11.988530    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sccld\" (UniqueName: \"kubernetes.io/projected/201466a6-58e5-49a6-869f-2161d2c8b708-kube-api-access-sccld\") pod \"ingress-canary-28f9z\" (UID: \"201466a6-58e5-49a6-869f-2161d2c8b708\") " pod="openshift-ingress-canary/ingress-canary-28f9z"
Jun 03 23:25:12.019879 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:12.019847    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-28f9z"
Jun 03 23:25:12.273559 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:12.273483    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:25:12.273721 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:12.273483    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:25:12.273721 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:12.273623    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:25:12.276331 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:12.276310    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jun 03 23:25:12.276573 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:12.276557    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jun 03 23:25:12.277064 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:12.277044    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jun 03 23:25:12.277258 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:12.277087    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-p9sbx\""
Jun 03 23:25:12.277258 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:12.277112    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jun 03 23:25:12.277445 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:12.277429    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8mqss\""
Jun 03 23:25:12.469640 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:12.469425    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-28f9z"]
Jun 03 23:25:12.473096 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:25:12.473071    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod201466a6_58e5_49a6_869f_2161d2c8b708.slice/crio-6938194857b966720c63174db1afb9e4a7505224aaf668ef723abc4925d5e62e WatchSource:0}: Error finding container 6938194857b966720c63174db1afb9e4a7505224aaf668ef723abc4925d5e62e: Status 404 returned error can't find the container with id 6938194857b966720c63174db1afb9e4a7505224aaf668ef723abc4925d5e62e
Jun 03 23:25:12.480297 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:12.480278    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:12.480387 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:12.480375    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 23:25:12.480448 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:12.480436    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls podName:3a1a6edf-938e-4784-bbc6-74c8829977a2 nodeName:}" failed. No retries permitted until 2026-06-03 23:25:13.480421302 +0000 UTC m=+35.643099355 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls") pod "dns-default-5q42g" (UID: "3a1a6edf-938e-4784-bbc6-74c8829977a2") : secret "dns-default-metrics-tls" not found
Jun 03 23:25:12.510442 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:12.510397    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-28f9z" event={"ID":"201466a6-58e5-49a6-869f-2161d2c8b708","Type":"ContainerStarted","Data":"6938194857b966720c63174db1afb9e4a7505224aaf668ef723abc4925d5e62e"}
Jun 03 23:25:13.487925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:13.487888    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:13.488396 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:13.488014    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 23:25:13.488396 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:13.488072    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls podName:3a1a6edf-938e-4784-bbc6-74c8829977a2 nodeName:}" failed. No retries permitted until 2026-06-03 23:25:15.488055176 +0000 UTC m=+37.650733249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls") pod "dns-default-5q42g" (UID: "3a1a6edf-938e-4784-bbc6-74c8829977a2") : secret "dns-default-metrics-tls" not found
Jun 03 23:25:13.515248 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:13.515202    2577 generic.go:356] "Generic (PLEG): container finished" podID="29474ba6-d997-4d27-a609-7adc008c79ce" containerID="d8235b6beaeb4334efaa013924f70ba2f5194bc69cd3a3724f7fa7049feb033e" exitCode=0
Jun 03 23:25:13.515248 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:13.515241    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rgddf" event={"ID":"29474ba6-d997-4d27-a609-7adc008c79ce","Type":"ContainerDied","Data":"d8235b6beaeb4334efaa013924f70ba2f5194bc69cd3a3724f7fa7049feb033e"}
Jun 03 23:25:14.521931 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:14.521899    2577 generic.go:356] "Generic (PLEG): container finished" podID="29474ba6-d997-4d27-a609-7adc008c79ce" containerID="02a364fedcae7bbcffced8df956e4959926365910c7b876835f8f4027569bd5c" exitCode=0
Jun 03 23:25:14.522663 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:14.521980    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rgddf" event={"ID":"29474ba6-d997-4d27-a609-7adc008c79ce","Type":"ContainerDied","Data":"02a364fedcae7bbcffced8df956e4959926365910c7b876835f8f4027569bd5c"}
Jun 03 23:25:14.523518 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:14.523497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-28f9z" event={"ID":"201466a6-58e5-49a6-869f-2161d2c8b708","Type":"ContainerStarted","Data":"07240863c38ea89097cb42cfe7c42d861c5f660e80fa1f45bc05853e79adc19a"}
Jun 03 23:25:15.506156 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:15.506126    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:15.506346 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:15.506278    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 23:25:15.506346 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:15.506346    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls podName:3a1a6edf-938e-4784-bbc6-74c8829977a2 nodeName:}" failed. No retries permitted until 2026-06-03 23:25:19.506329961 +0000 UTC m=+41.669008028 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls") pod "dns-default-5q42g" (UID: "3a1a6edf-938e-4784-bbc6-74c8829977a2") : secret "dns-default-metrics-tls" not found
Jun 03 23:25:15.529479 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:15.529430    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rgddf" event={"ID":"29474ba6-d997-4d27-a609-7adc008c79ce","Type":"ContainerStarted","Data":"c5fab4d05eaf4e10a6b233258f2d3f80150e81cc2211fa9412e070ef8e4a9293"}
Jun 03 23:25:15.554818 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:15.554762    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rgddf" podStartSLOduration=8.916431322 podStartE2EDuration="37.554742425s" totalImagesPullingTime="28.638311103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-03 23:24:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:25:15.554661506 +0000 UTC m=+37.717339581" watchObservedRunningTime="2026-06-03 23:25:15.554742425 +0000 UTC m=+37.717420503"
Jun 03 23:25:15.555304 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:15.555114    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-28f9z" podStartSLOduration=2.664307181 podStartE2EDuration="4.555102174s" totalImagesPullingTime="1.890794993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:25:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:25:14.566447277 +0000 UTC m=+36.729125354" watchObservedRunningTime="2026-06-03 23:25:15.555102174 +0000 UTC m=+37.717780252"
Jun 03 23:25:19.534858 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:19.534659    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:19.535268 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:19.534810    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 23:25:19.535268 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:19.534924    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls podName:3a1a6edf-938e-4784-bbc6-74c8829977a2 nodeName:}" failed. No retries permitted until 2026-06-03 23:25:27.534908946 +0000 UTC m=+49.697586998 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls") pod "dns-default-5q42g" (UID: "3a1a6edf-938e-4784-bbc6-74c8829977a2") : secret "dns-default-metrics-tls" not found
Jun 03 23:25:23.821684 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:23.821650    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-784dcbddc6-97bdt"]
Jun 03 23:25:23.855083 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:23.855054    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-784dcbddc6-97bdt"]
Jun 03 23:25:23.855236 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:23.855144    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-784dcbddc6-97bdt"
Jun 03 23:25:23.857481 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:23.857457    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jun 03 23:25:23.858586 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:23.858560    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jun 03 23:25:23.858695 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:23.858593    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jun 03 23:25:23.858695 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:23.858643    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jun 03 23:25:23.858805 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:23.858721    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-2nq65\""
Jun 03 23:25:23.962103 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:23.962067    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn2j5\" (UniqueName: \"kubernetes.io/projected/c014c19d-a39b-4f2d-9b13-43a965ff282e-kube-api-access-xn2j5\") pod \"managed-serviceaccount-addon-agent-784dcbddc6-97bdt\" (UID: \"c014c19d-a39b-4f2d-9b13-43a965ff282e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-784dcbddc6-97bdt"
Jun 03 23:25:23.962278 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:23.962120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c014c19d-a39b-4f2d-9b13-43a965ff282e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-784dcbddc6-97bdt\" (UID: \"c014c19d-a39b-4f2d-9b13-43a965ff282e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-784dcbddc6-97bdt"
Jun 03 23:25:24.063356 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:24.063328    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c014c19d-a39b-4f2d-9b13-43a965ff282e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-784dcbddc6-97bdt\" (UID: \"c014c19d-a39b-4f2d-9b13-43a965ff282e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-784dcbddc6-97bdt"
Jun 03 23:25:24.063448 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:24.063399    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn2j5\" (UniqueName: \"kubernetes.io/projected/c014c19d-a39b-4f2d-9b13-43a965ff282e-kube-api-access-xn2j5\") pod \"managed-serviceaccount-addon-agent-784dcbddc6-97bdt\" (UID: \"c014c19d-a39b-4f2d-9b13-43a965ff282e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-784dcbddc6-97bdt"
Jun 03 23:25:24.067177 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:24.067149    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c014c19d-a39b-4f2d-9b13-43a965ff282e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-784dcbddc6-97bdt\" (UID: \"c014c19d-a39b-4f2d-9b13-43a965ff282e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-784dcbddc6-97bdt"
Jun 03 23:25:24.086048 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:24.085994    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn2j5\" (UniqueName: \"kubernetes.io/projected/c014c19d-a39b-4f2d-9b13-43a965ff282e-kube-api-access-xn2j5\") pod \"managed-serviceaccount-addon-agent-784dcbddc6-97bdt\" (UID: \"c014c19d-a39b-4f2d-9b13-43a965ff282e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-784dcbddc6-97bdt"
Jun 03 23:25:24.175632 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:24.175597    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-784dcbddc6-97bdt"
Jun 03 23:25:24.315321 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:24.315289    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-784dcbddc6-97bdt"]
Jun 03 23:25:24.318747 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:25:24.318714    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc014c19d_a39b_4f2d_9b13_43a965ff282e.slice/crio-b1e2acaacf5c344e6f200267e99ee298b78f9e33c152570e37b38d9e1aff55ef WatchSource:0}: Error finding container b1e2acaacf5c344e6f200267e99ee298b78f9e33c152570e37b38d9e1aff55ef: Status 404 returned error can't find the container with id b1e2acaacf5c344e6f200267e99ee298b78f9e33c152570e37b38d9e1aff55ef
Jun 03 23:25:24.550650 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:24.550568    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-784dcbddc6-97bdt" event={"ID":"c014c19d-a39b-4f2d-9b13-43a965ff282e","Type":"ContainerStarted","Data":"b1e2acaacf5c344e6f200267e99ee298b78f9e33c152570e37b38d9e1aff55ef"}
Jun 03 23:25:27.587651 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:27.587624    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:27.587947 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:27.587725    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 23:25:27.587947 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:27.587768    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls podName:3a1a6edf-938e-4784-bbc6-74c8829977a2 nodeName:}" failed. No retries permitted until 2026-06-03 23:25:43.587756377 +0000 UTC m=+65.750434429 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls") pod "dns-default-5q42g" (UID: "3a1a6edf-938e-4784-bbc6-74c8829977a2") : secret "dns-default-metrics-tls" not found
Jun 03 23:25:28.561726 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:28.561687    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-784dcbddc6-97bdt" event={"ID":"c014c19d-a39b-4f2d-9b13-43a965ff282e","Type":"ContainerStarted","Data":"7e2b2e4a7e531879e552a0e94f0851ac3033918ebf7b14802617eb64c89510eb"}
Jun 03 23:25:28.578523 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:28.578475    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-784dcbddc6-97bdt" podStartSLOduration=2.328215134 podStartE2EDuration="5.578457168s" totalImagesPullingTime="3.250242034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:25:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:25:28.577834631 +0000 UTC m=+50.740512704" watchObservedRunningTime="2026-06-03 23:25:28.578457168 +0000 UTC m=+50.741135242"
Jun 03 23:25:37.506855 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:37.506828    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4s5gr"
Jun 03 23:25:42.986221 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:42.986184    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs\") pod \"network-metrics-daemon-ssbpc\" (UID: \"81ee1ff9-1018-4eaf-b246-18b5f7a95f78\") " pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:25:42.988704 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:42.988687    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jun 03 23:25:42.997162 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:42.997143    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jun 03 23:25:42.997225 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:42.997198    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs podName:81ee1ff9-1018-4eaf-b246-18b5f7a95f78 nodeName:}" failed. No retries permitted until 2026-06-03 23:26:46.997182928 +0000 UTC m=+129.159860981 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs") pod "network-metrics-daemon-ssbpc" (UID: "81ee1ff9-1018-4eaf-b246-18b5f7a95f78") : secret "metrics-daemon-secret" not found
Jun 03 23:25:43.086777 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:43.086751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret\") pod \"global-pull-secret-syncer-9d2cb\" (UID: \"55ceda9f-6200-4a21-a050-d290d631043f\") " pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:25:43.086878 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:43.086795    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dggl\" (UniqueName: \"kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl\") pod \"network-check-target-9gpsw\" (UID: \"7ab2d105-78a7-4089-9f3b-f0b03411ec50\") " pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:25:43.089781 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:43.089763    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jun 03 23:25:43.089835 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:43.089763    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jun 03 23:25:43.099763 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:43.099724    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55ceda9f-6200-4a21-a050-d290d631043f-original-pull-secret\") pod \"global-pull-secret-syncer-9d2cb\" (UID: \"55ceda9f-6200-4a21-a050-d290d631043f\") " pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:25:43.099869 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:43.099801    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jun 03 23:25:43.110575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:43.110552    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dggl\" (UniqueName: \"kubernetes.io/projected/7ab2d105-78a7-4089-9f3b-f0b03411ec50-kube-api-access-6dggl\") pod \"network-check-target-9gpsw\" (UID: \"7ab2d105-78a7-4089-9f3b-f0b03411ec50\") " pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:25:43.187448 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:43.187414    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-p9sbx\""
Jun 03 23:25:43.195529 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:43.195509    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:25:43.199749 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:43.199724    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9d2cb"
Jun 03 23:25:43.379846 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:43.379817    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9gpsw"]
Jun 03 23:25:43.382717 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:25:43.382689    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ab2d105_78a7_4089_9f3b_f0b03411ec50.slice/crio-802433a5ac3eebfead054bef32689808f1a03b5bd9557bd2c7e48b8fddf418fd WatchSource:0}: Error finding container 802433a5ac3eebfead054bef32689808f1a03b5bd9557bd2c7e48b8fddf418fd: Status 404 returned error can't find the container with id 802433a5ac3eebfead054bef32689808f1a03b5bd9557bd2c7e48b8fddf418fd
Jun 03 23:25:43.391098 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:43.391077    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9d2cb"]
Jun 03 23:25:43.393904 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:25:43.393878    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55ceda9f_6200_4a21_a050_d290d631043f.slice/crio-d646b3d3c84536552784918d6a14e6026d2d492f86881334318abc339a737cc5 WatchSource:0}: Error finding container d646b3d3c84536552784918d6a14e6026d2d492f86881334318abc339a737cc5: Status 404 returned error can't find the container with id d646b3d3c84536552784918d6a14e6026d2d492f86881334318abc339a737cc5
Jun 03 23:25:43.590797 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:43.590766    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:25:43.590913 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:43.590894    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 23:25:43.590956 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:25:43.590942    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls podName:3a1a6edf-938e-4784-bbc6-74c8829977a2 nodeName:}" failed. No retries permitted until 2026-06-03 23:26:15.590929836 +0000 UTC m=+97.753607889 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls") pod "dns-default-5q42g" (UID: "3a1a6edf-938e-4784-bbc6-74c8829977a2") : secret "dns-default-metrics-tls" not found
Jun 03 23:25:43.598492 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:43.598466    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9d2cb" event={"ID":"55ceda9f-6200-4a21-a050-d290d631043f","Type":"ContainerStarted","Data":"d646b3d3c84536552784918d6a14e6026d2d492f86881334318abc339a737cc5"}
Jun 03 23:25:43.599511 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:43.599488    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9gpsw" event={"ID":"7ab2d105-78a7-4089-9f3b-f0b03411ec50","Type":"ContainerStarted","Data":"802433a5ac3eebfead054bef32689808f1a03b5bd9557bd2c7e48b8fddf418fd"}
Jun 03 23:25:48.613986 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:48.613949    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9d2cb" event={"ID":"55ceda9f-6200-4a21-a050-d290d631043f","Type":"ContainerStarted","Data":"2521ec28cf52dba272e3736240115a0f20eeb5ac459a460230dcb8119260ee99"}
Jun 03 23:25:48.615457 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:48.615431    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9gpsw" event={"ID":"7ab2d105-78a7-4089-9f3b-f0b03411ec50","Type":"ContainerStarted","Data":"8ca09061d1f2fa38358aeb4393c03eed8c7b087612793a1b7c4bcd3bd19b5d3b"}
Jun 03 23:25:48.615599 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:48.615549    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:25:48.644796 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:48.644751    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9d2cb" podStartSLOduration=66.34958208 podStartE2EDuration="1m10.644736287s" totalImagesPullingTime="4.295154207s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-03 23:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:25:48.630151204 +0000 UTC m=+70.792829280" watchObservedRunningTime="2026-06-03 23:25:48.644736287 +0000 UTC m=+70.807414338"
Jun 03 23:25:48.645177 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:25:48.645143    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9gpsw" podStartSLOduration=66.343625979 podStartE2EDuration="1m10.645132543s" totalImagesPullingTime="4.301506564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:25:48.644633277 +0000 UTC m=+70.807311350" watchObservedRunningTime="2026-06-03 23:25:48.645132543 +0000 UTC m=+70.807810620"
Jun 03 23:26:15.686386 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:26:15.686264    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:26:15.686856 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:26:15.686411    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 23:26:15.686856 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:26:15.686489    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls podName:3a1a6edf-938e-4784-bbc6-74c8829977a2 nodeName:}" failed. No retries permitted until 2026-06-03 23:27:19.686474737 +0000 UTC m=+161.849152789 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls") pod "dns-default-5q42g" (UID: "3a1a6edf-938e-4784-bbc6-74c8829977a2") : secret "dns-default-metrics-tls" not found
Jun 03 23:26:19.620445 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:26:19.620416    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9gpsw"
Jun 03 23:26:46.999724 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:26:46.999680    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs\") pod \"network-metrics-daemon-ssbpc\" (UID: \"81ee1ff9-1018-4eaf-b246-18b5f7a95f78\") " pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:26:47.000194 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:26:46.999844    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jun 03 23:26:47.000194 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:26:46.999911    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs podName:81ee1ff9-1018-4eaf-b246-18b5f7a95f78 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:48.999894792 +0000 UTC m=+251.162572845 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs") pod "network-metrics-daemon-ssbpc" (UID: "81ee1ff9-1018-4eaf-b246-18b5f7a95f78") : secret "metrics-daemon-secret" not found
Jun 03 23:27:14.713337 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:27:14.713296    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5q42g" podUID="3a1a6edf-938e-4784-bbc6-74c8829977a2"
Jun 03 23:27:14.813364 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:27:14.813340    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5q42g"
Jun 03 23:27:15.293545 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:27:15.293496    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ssbpc" podUID="81ee1ff9-1018-4eaf-b246-18b5f7a95f78"
Jun 03 23:27:19.718024 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:27:19.717991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:27:19.718522 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:27:19.718180    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 23:27:19.718522 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:27:19.718290    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls podName:3a1a6edf-938e-4784-bbc6-74c8829977a2 nodeName:}" failed. No retries permitted until 2026-06-03 23:29:21.718266374 +0000 UTC m=+283.880944428 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls") pod "dns-default-5q42g" (UID: "3a1a6edf-938e-4784-bbc6-74c8829977a2") : secret "dns-default-metrics-tls" not found
Jun 03 23:27:26.272951 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:27:26.272864    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:27:27.844327 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:27:27.844299    2577 generic.go:356] "Generic (PLEG): container finished" podID="c014c19d-a39b-4f2d-9b13-43a965ff282e" containerID="7e2b2e4a7e531879e552a0e94f0851ac3033918ebf7b14802617eb64c89510eb" exitCode=255
Jun 03 23:27:27.844686 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:27:27.844348    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-784dcbddc6-97bdt" event={"ID":"c014c19d-a39b-4f2d-9b13-43a965ff282e","Type":"ContainerDied","Data":"7e2b2e4a7e531879e552a0e94f0851ac3033918ebf7b14802617eb64c89510eb"}
Jun 03 23:27:27.850095 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:27:27.850074    2577 scope.go:117] "RemoveContainer" containerID="7e2b2e4a7e531879e552a0e94f0851ac3033918ebf7b14802617eb64c89510eb"
Jun 03 23:27:28.848294 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:27:28.848253    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-784dcbddc6-97bdt" event={"ID":"c014c19d-a39b-4f2d-9b13-43a965ff282e","Type":"ContainerStarted","Data":"98bd205c59a0ee9344ad4fefeee86a4255ec37bfdcad4eff29e3de144e0831f1"}
Jun 03 23:28:14.812993 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.812940    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-55b9b48c7c-jw6cm"]
Jun 03 23:28:14.814769 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.814752    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:14.818881 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.818860    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-pvpbs\""
Jun 03 23:28:14.820285 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.820266    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jun 03 23:28:14.820497 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.820482    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jun 03 23:28:14.821058 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.820866    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jun 03 23:28:14.821273 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.821255    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jun 03 23:28:14.821631 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.821613    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jun 03 23:28:14.822141 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.822125    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jun 03 23:28:14.843808 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.843762    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-55b9b48c7c-jw6cm"]
Jun 03 23:28:14.881816 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.881788    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9d8l\" (UniqueName: \"kubernetes.io/projected/98372747-9a9e-4a1e-ad2f-d274d1db6889-kube-api-access-w9d8l\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:14.881988 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.881828    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:14.881988 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.881847    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:14.881988 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.881870    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-default-certificate\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:14.881988 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.881945    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-stats-auth\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:14.913750 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.913720    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-pk2xd"]
Jun 03 23:28:14.915589 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.915566    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-pk2xd"
Jun 03 23:28:14.915904 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.915883    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"]
Jun 03 23:28:14.917426 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.917406    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:14.917957 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.917920    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78d9cb695b-4hdm9"]
Jun 03 23:28:14.919126 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.919106    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jun 03 23:28:14.919501 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.919480    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-pmpxk\""
Jun 03 23:28:14.919594 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.919575    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jun 03 23:28:14.919645 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.919593    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jun 03 23:28:14.919939 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.919911    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:14.920396 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.920377    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jun 03 23:28:14.920535 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.920520    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jun 03 23:28:14.920657 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.920643    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-69w9s\""
Jun 03 23:28:14.920738 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.920724    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jun 03 23:28:14.923085 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.923067    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jun 03 23:28:14.923166 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.923140    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jun 03 23:28:14.923307 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.923293    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jun 03 23:28:14.923489 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.923474    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4x8v9\""
Jun 03 23:28:14.930134 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.929754    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jun 03 23:28:14.930134 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.929929    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-pk2xd"]
Jun 03 23:28:14.930927 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.930906    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"]
Jun 03 23:28:14.934642 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.934621    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78d9cb695b-4hdm9"]
Jun 03 23:28:14.982259 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.982230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c9b06e42-99fa-47e2-847e-47b427f82c56-ca-trust-extracted\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:14.982413 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.982271    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9d8l\" (UniqueName: \"kubernetes.io/projected/98372747-9a9e-4a1e-ad2f-d274d1db6889-kube-api-access-w9d8l\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:14.982413 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.982319    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:14.982678 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.982346    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6r5z\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-kube-api-access-x6r5z\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:14.982814 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.982693    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:14.982814 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.982743    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-default-certificate\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:14.982918 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.982770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-telemetry-config\") pod \"cluster-monitoring-operator-57769895dd-bhxbc\" (UID: \"6125ef6f-2e47-4f8c-9668-274bdffe1ca2\") " pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:14.982981 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.982961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-57769895dd-bhxbc\" (UID: \"6125ef6f-2e47-4f8c-9668-274bdffe1ca2\") " pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:14.983040 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:14.983008    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jun 03 23:28:14.983089 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.983058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-stats-auth\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:14.983138 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.983091    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c9b06e42-99fa-47e2-847e-47b427f82c56-trusted-ca\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:14.983188 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.983149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdb8b\" (UniqueName: \"kubernetes.io/projected/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-kube-api-access-rdb8b\") pod \"cluster-monitoring-operator-57769895dd-bhxbc\" (UID: \"6125ef6f-2e47-4f8c-9668-274bdffe1ca2\") " pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:14.983346 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.983281    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4chz4\" (UniqueName: \"kubernetes.io/projected/0a55c225-4fd3-4caa-b9c8-1f535f65637f-kube-api-access-4chz4\") pod \"volume-data-source-validator-78dd696465-pk2xd\" (UID: \"0a55c225-4fd3-4caa-b9c8-1f535f65637f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-pk2xd"
Jun 03 23:28:14.983590 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.983538    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c9b06e42-99fa-47e2-847e-47b427f82c56-image-registry-private-configuration\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:14.983687 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.983614    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:14.983781 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.983761    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-certificates\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:14.983887 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.983842    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c9b06e42-99fa-47e2-847e-47b427f82c56-installation-pull-secrets\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:14.983958 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:14.983891    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle podName:98372747-9a9e-4a1e-ad2f-d274d1db6889 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:15.483834215 +0000 UTC m=+217.646512287 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle") pod "router-default-55b9b48c7c-jw6cm" (UID: "98372747-9a9e-4a1e-ad2f-d274d1db6889") : configmap references non-existent config key: service-ca.crt
Jun 03 23:28:14.984128 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.984107    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-bound-sa-token\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:14.984407 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:14.984386    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs podName:98372747-9a9e-4a1e-ad2f-d274d1db6889 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:15.484284123 +0000 UTC m=+217.646962182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs") pod "router-default-55b9b48c7c-jw6cm" (UID: "98372747-9a9e-4a1e-ad2f-d274d1db6889") : secret "router-metrics-certs-default" not found
Jun 03 23:28:14.987541 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.987518    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-stats-auth\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:14.988650 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.988627    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-default-certificate\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:14.990412 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:14.990391    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9d8l\" (UniqueName: \"kubernetes.io/projected/98372747-9a9e-4a1e-ad2f-d274d1db6889-kube-api-access-w9d8l\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:15.008575 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.008548    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85bd8fd97d-tsn6l"]
Jun 03 23:28:15.010357 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.010343    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:15.012479 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.012447    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jun 03 23:28:15.012576 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.012451    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jun 03 23:28:15.012770 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.012752    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-k55bc\""
Jun 03 23:28:15.012852 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.012782    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jun 03 23:28:15.012917 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.012879    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jun 03 23:28:15.018773 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.018752    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jun 03 23:28:15.021415 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.021397    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85bd8fd97d-tsn6l"]
Jun 03 23:28:15.085393 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.085370    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:15.085525 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.085402    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcsxd\" (UniqueName: \"kubernetes.io/projected/397370f1-b682-48ca-b643-f447f300c2a5-kube-api-access-xcsxd\") pod \"console-operator-85bd8fd97d-tsn6l\" (UID: \"397370f1-b682-48ca-b643-f447f300c2a5\") " pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:15.085525 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.085422    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/397370f1-b682-48ca-b643-f447f300c2a5-trusted-ca\") pod \"console-operator-85bd8fd97d-tsn6l\" (UID: \"397370f1-b682-48ca-b643-f447f300c2a5\") " pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:15.085525 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.085438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c9b06e42-99fa-47e2-847e-47b427f82c56-installation-pull-secrets\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:15.085525 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.085453    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-bound-sa-token\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:15.085730 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:15.085557    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 03 23:28:15.085730 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:15.085575    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78d9cb695b-4hdm9: secret "image-registry-tls" not found
Jun 03 23:28:15.085730 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.085619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-telemetry-config\") pod \"cluster-monitoring-operator-57769895dd-bhxbc\" (UID: \"6125ef6f-2e47-4f8c-9668-274bdffe1ca2\") " pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:15.085730 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:15.085633    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls podName:c9b06e42-99fa-47e2-847e-47b427f82c56 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:15.585619407 +0000 UTC m=+217.748297459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls") pod "image-registry-78d9cb695b-4hdm9" (UID: "c9b06e42-99fa-47e2-847e-47b427f82c56") : secret "image-registry-tls" not found
Jun 03 23:28:15.085730 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.085676    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-57769895dd-bhxbc\" (UID: \"6125ef6f-2e47-4f8c-9668-274bdffe1ca2\") " pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:15.085730 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.085705    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/397370f1-b682-48ca-b643-f447f300c2a5-config\") pod \"console-operator-85bd8fd97d-tsn6l\" (UID: \"397370f1-b682-48ca-b643-f447f300c2a5\") " pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:15.085989 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.085732    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c9b06e42-99fa-47e2-847e-47b427f82c56-trusted-ca\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:15.085989 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.085758    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdb8b\" (UniqueName: \"kubernetes.io/projected/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-kube-api-access-rdb8b\") pod \"cluster-monitoring-operator-57769895dd-bhxbc\" (UID: \"6125ef6f-2e47-4f8c-9668-274bdffe1ca2\") " pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:15.085989 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:15.085777    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 03 23:28:15.085989 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:15.085821    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls podName:6125ef6f-2e47-4f8c-9668-274bdffe1ca2 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:15.585805086 +0000 UTC m=+217.748483139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-57769895dd-bhxbc" (UID: "6125ef6f-2e47-4f8c-9668-274bdffe1ca2") : secret "cluster-monitoring-operator-tls" not found
Jun 03 23:28:15.085989 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.085922    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4chz4\" (UniqueName: \"kubernetes.io/projected/0a55c225-4fd3-4caa-b9c8-1f535f65637f-kube-api-access-4chz4\") pod \"volume-data-source-validator-78dd696465-pk2xd\" (UID: \"0a55c225-4fd3-4caa-b9c8-1f535f65637f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-pk2xd"
Jun 03 23:28:15.085989 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.085955    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c9b06e42-99fa-47e2-847e-47b427f82c56-image-registry-private-configuration\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:15.085989 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.085986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-certificates\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:15.086348 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.086025    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c9b06e42-99fa-47e2-847e-47b427f82c56-ca-trust-extracted\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:15.086348 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.086074    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6r5z\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-kube-api-access-x6r5z\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:15.086348 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.086119    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/397370f1-b682-48ca-b643-f447f300c2a5-serving-cert\") pod \"console-operator-85bd8fd97d-tsn6l\" (UID: \"397370f1-b682-48ca-b643-f447f300c2a5\") " pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:15.086504 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.086388    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-telemetry-config\") pod \"cluster-monitoring-operator-57769895dd-bhxbc\" (UID: \"6125ef6f-2e47-4f8c-9668-274bdffe1ca2\") " pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:15.086504 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.086427    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c9b06e42-99fa-47e2-847e-47b427f82c56-ca-trust-extracted\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:15.086822 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.086801    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c9b06e42-99fa-47e2-847e-47b427f82c56-trusted-ca\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:15.086882 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.086822    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-certificates\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:15.087998 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.087982    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c9b06e42-99fa-47e2-847e-47b427f82c56-installation-pull-secrets\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:15.088183 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.088168    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c9b06e42-99fa-47e2-847e-47b427f82c56-image-registry-private-configuration\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:15.093438 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.093416    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-bound-sa-token\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:15.093921 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.093898    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4chz4\" (UniqueName: \"kubernetes.io/projected/0a55c225-4fd3-4caa-b9c8-1f535f65637f-kube-api-access-4chz4\") pod \"volume-data-source-validator-78dd696465-pk2xd\" (UID: \"0a55c225-4fd3-4caa-b9c8-1f535f65637f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-pk2xd"
Jun 03 23:28:15.094121 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.094098    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6r5z\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-kube-api-access-x6r5z\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:15.094296 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.094273    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdb8b\" (UniqueName: \"kubernetes.io/projected/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-kube-api-access-rdb8b\") pod \"cluster-monitoring-operator-57769895dd-bhxbc\" (UID: \"6125ef6f-2e47-4f8c-9668-274bdffe1ca2\") " pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:15.186866 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.186837    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/397370f1-b682-48ca-b643-f447f300c2a5-serving-cert\") pod \"console-operator-85bd8fd97d-tsn6l\" (UID: \"397370f1-b682-48ca-b643-f447f300c2a5\") " pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:15.186990 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.186969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcsxd\" (UniqueName: \"kubernetes.io/projected/397370f1-b682-48ca-b643-f447f300c2a5-kube-api-access-xcsxd\") pod \"console-operator-85bd8fd97d-tsn6l\" (UID: \"397370f1-b682-48ca-b643-f447f300c2a5\") " pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:15.187057 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.186999    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/397370f1-b682-48ca-b643-f447f300c2a5-trusted-ca\") pod \"console-operator-85bd8fd97d-tsn6l\" (UID: \"397370f1-b682-48ca-b643-f447f300c2a5\") " pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:15.187057 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.187044    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/397370f1-b682-48ca-b643-f447f300c2a5-config\") pod \"console-operator-85bd8fd97d-tsn6l\" (UID: \"397370f1-b682-48ca-b643-f447f300c2a5\") " pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:15.187593 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.187569    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/397370f1-b682-48ca-b643-f447f300c2a5-config\") pod \"console-operator-85bd8fd97d-tsn6l\" (UID: \"397370f1-b682-48ca-b643-f447f300c2a5\") " pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:15.187816 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.187798    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/397370f1-b682-48ca-b643-f447f300c2a5-trusted-ca\") pod \"console-operator-85bd8fd97d-tsn6l\" (UID: \"397370f1-b682-48ca-b643-f447f300c2a5\") " pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:15.188806 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.188788    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/397370f1-b682-48ca-b643-f447f300c2a5-serving-cert\") pod \"console-operator-85bd8fd97d-tsn6l\" (UID: \"397370f1-b682-48ca-b643-f447f300c2a5\") " pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:15.194432 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.194412    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcsxd\" (UniqueName: \"kubernetes.io/projected/397370f1-b682-48ca-b643-f447f300c2a5-kube-api-access-xcsxd\") pod \"console-operator-85bd8fd97d-tsn6l\" (UID: \"397370f1-b682-48ca-b643-f447f300c2a5\") " pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:15.226250 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.226231    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-pk2xd"
Jun 03 23:28:15.320509 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.320480    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:15.345161 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.345077    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-pk2xd"]
Jun 03 23:28:15.349177 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:28:15.349152    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a55c225_4fd3_4caa_b9c8_1f535f65637f.slice/crio-c8aab77e65f34daa1200397fec5fa0ed47aaa2e4ae241266b30e862f0288b6dd WatchSource:0}: Error finding container c8aab77e65f34daa1200397fec5fa0ed47aaa2e4ae241266b30e862f0288b6dd: Status 404 returned error can't find the container with id c8aab77e65f34daa1200397fec5fa0ed47aaa2e4ae241266b30e862f0288b6dd
Jun 03 23:28:15.441562 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.441527    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85bd8fd97d-tsn6l"]
Jun 03 23:28:15.444279 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:28:15.444256    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod397370f1_b682_48ca_b643_f447f300c2a5.slice/crio-a9fba875579c7d78d85e4ed4cbe66c62be621f2f532e27e3f5d86b3928c6cf26 WatchSource:0}: Error finding container a9fba875579c7d78d85e4ed4cbe66c62be621f2f532e27e3f5d86b3928c6cf26: Status 404 returned error can't find the container with id a9fba875579c7d78d85e4ed4cbe66c62be621f2f532e27e3f5d86b3928c6cf26
Jun 03 23:28:15.489170 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.489132    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:15.489170 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.489176    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:15.489421 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:15.489313    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle podName:98372747-9a9e-4a1e-ad2f-d274d1db6889 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:16.489292922 +0000 UTC m=+218.651970983 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle") pod "router-default-55b9b48c7c-jw6cm" (UID: "98372747-9a9e-4a1e-ad2f-d274d1db6889") : configmap references non-existent config key: service-ca.crt
Jun 03 23:28:15.489421 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:15.489332    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jun 03 23:28:15.489421 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:15.489372    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs podName:98372747-9a9e-4a1e-ad2f-d274d1db6889 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:16.489359461 +0000 UTC m=+218.652037513 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs") pod "router-default-55b9b48c7c-jw6cm" (UID: "98372747-9a9e-4a1e-ad2f-d274d1db6889") : secret "router-metrics-certs-default" not found
Jun 03 23:28:15.590029 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.589992    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:15.590188 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.590062    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-57769895dd-bhxbc\" (UID: \"6125ef6f-2e47-4f8c-9668-274bdffe1ca2\") " pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:15.590188 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:15.590146    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 03 23:28:15.590188 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:15.590165    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78d9cb695b-4hdm9: secret "image-registry-tls" not found
Jun 03 23:28:15.590323 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:15.590219    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 03 23:28:15.590323 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:15.590240    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls podName:c9b06e42-99fa-47e2-847e-47b427f82c56 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:16.590224584 +0000 UTC m=+218.752902652 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls") pod "image-registry-78d9cb695b-4hdm9" (UID: "c9b06e42-99fa-47e2-847e-47b427f82c56") : secret "image-registry-tls" not found
Jun 03 23:28:15.590323 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:15.590274    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls podName:6125ef6f-2e47-4f8c-9668-274bdffe1ca2 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:16.590261154 +0000 UTC m=+218.752939206 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-57769895dd-bhxbc" (UID: "6125ef6f-2e47-4f8c-9668-274bdffe1ca2") : secret "cluster-monitoring-operator-tls" not found
Jun 03 23:28:15.963290 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.963252    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-pk2xd" event={"ID":"0a55c225-4fd3-4caa-b9c8-1f535f65637f","Type":"ContainerStarted","Data":"c8aab77e65f34daa1200397fec5fa0ed47aaa2e4ae241266b30e862f0288b6dd"}
Jun 03 23:28:15.964495 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:15.964467    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l" event={"ID":"397370f1-b682-48ca-b643-f447f300c2a5","Type":"ContainerStarted","Data":"a9fba875579c7d78d85e4ed4cbe66c62be621f2f532e27e3f5d86b3928c6cf26"}
Jun 03 23:28:16.497888 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:16.497851    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:16.498047 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:16.497901    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:16.498047 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:16.498020    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jun 03 23:28:16.498150 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:16.498024    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle podName:98372747-9a9e-4a1e-ad2f-d274d1db6889 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:18.498005723 +0000 UTC m=+220.660683774 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle") pod "router-default-55b9b48c7c-jw6cm" (UID: "98372747-9a9e-4a1e-ad2f-d274d1db6889") : configmap references non-existent config key: service-ca.crt
Jun 03 23:28:16.498240 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:16.498182    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs podName:98372747-9a9e-4a1e-ad2f-d274d1db6889 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:18.498160397 +0000 UTC m=+220.660838457 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs") pod "router-default-55b9b48c7c-jw6cm" (UID: "98372747-9a9e-4a1e-ad2f-d274d1db6889") : secret "router-metrics-certs-default" not found
Jun 03 23:28:16.599423 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:16.599334    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-57769895dd-bhxbc\" (UID: \"6125ef6f-2e47-4f8c-9668-274bdffe1ca2\") " pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:16.599636 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:16.599452    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:16.599636 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:16.599606    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 03 23:28:16.599636 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:16.599621    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78d9cb695b-4hdm9: secret "image-registry-tls" not found
Jun 03 23:28:16.600029 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:16.599678    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls podName:c9b06e42-99fa-47e2-847e-47b427f82c56 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:18.599658926 +0000 UTC m=+220.762336980 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls") pod "image-registry-78d9cb695b-4hdm9" (UID: "c9b06e42-99fa-47e2-847e-47b427f82c56") : secret "image-registry-tls" not found
Jun 03 23:28:16.600118 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:16.600077    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 03 23:28:16.600184 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:16.600156    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls podName:6125ef6f-2e47-4f8c-9668-274bdffe1ca2 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:18.6001379 +0000 UTC m=+220.762815966 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-57769895dd-bhxbc" (UID: "6125ef6f-2e47-4f8c-9668-274bdffe1ca2") : secret "cluster-monitoring-operator-tls" not found
Jun 03 23:28:16.969366 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:16.969272    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-pk2xd" event={"ID":"0a55c225-4fd3-4caa-b9c8-1f535f65637f","Type":"ContainerStarted","Data":"1289dad8e358dcae45bf4c149df032fedf1088aff8c22c3e2d9c2b4d4407a563"}
Jun 03 23:28:16.983639 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:16.983586    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-pk2xd" podStartSLOduration=2.01569913 podStartE2EDuration="2.983567566s" totalImagesPullingTime="967.868436ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:28:16.982933806 +0000 UTC m=+219.145611890" watchObservedRunningTime="2026-06-03 23:28:16.983567566 +0000 UTC m=+219.146245664"
Jun 03 23:28:17.974531 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:17.974454    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/0.log"
Jun 03 23:28:17.974531 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:17.974501    2577 generic.go:356] "Generic (PLEG): container finished" podID="397370f1-b682-48ca-b643-f447f300c2a5" containerID="b67f93dad7b43928ddb82d3c268e8ac4f32f84df7cabc1e1f4ba8eea0fe2d9dd" exitCode=255
Jun 03 23:28:17.974934 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:17.974583    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l" event={"ID":"397370f1-b682-48ca-b643-f447f300c2a5","Type":"ContainerDied","Data":"b67f93dad7b43928ddb82d3c268e8ac4f32f84df7cabc1e1f4ba8eea0fe2d9dd"}
Jun 03 23:28:17.974934 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:17.974811    2577 scope.go:117] "RemoveContainer" containerID="b67f93dad7b43928ddb82d3c268e8ac4f32f84df7cabc1e1f4ba8eea0fe2d9dd"
Jun 03 23:28:18.515638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:18.515601    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:18.515638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:18.515642    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:18.515820 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:18.515747    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jun 03 23:28:18.515820 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:18.515784    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle podName:98372747-9a9e-4a1e-ad2f-d274d1db6889 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:22.515765345 +0000 UTC m=+224.678443416 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle") pod "router-default-55b9b48c7c-jw6cm" (UID: "98372747-9a9e-4a1e-ad2f-d274d1db6889") : configmap references non-existent config key: service-ca.crt
Jun 03 23:28:18.515820 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:18.515811    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs podName:98372747-9a9e-4a1e-ad2f-d274d1db6889 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:22.515803848 +0000 UTC m=+224.678481899 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs") pod "router-default-55b9b48c7c-jw6cm" (UID: "98372747-9a9e-4a1e-ad2f-d274d1db6889") : secret "router-metrics-certs-default" not found
Jun 03 23:28:18.616544 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:18.616510    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:18.616727 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:18.616575    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-57769895dd-bhxbc\" (UID: \"6125ef6f-2e47-4f8c-9668-274bdffe1ca2\") " pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:18.616727 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:18.616665    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 03 23:28:18.616727 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:18.616687    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78d9cb695b-4hdm9: secret "image-registry-tls" not found
Jun 03 23:28:18.616727 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:18.616699    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 03 23:28:18.616895 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:18.616755    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls podName:c9b06e42-99fa-47e2-847e-47b427f82c56 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:22.616735004 +0000 UTC m=+224.779413056 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls") pod "image-registry-78d9cb695b-4hdm9" (UID: "c9b06e42-99fa-47e2-847e-47b427f82c56") : secret "image-registry-tls" not found
Jun 03 23:28:18.616895 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:18.616777    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls podName:6125ef6f-2e47-4f8c-9668-274bdffe1ca2 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:22.616766746 +0000 UTC m=+224.779444802 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-57769895dd-bhxbc" (UID: "6125ef6f-2e47-4f8c-9668-274bdffe1ca2") : secret "cluster-monitoring-operator-tls" not found
Jun 03 23:28:18.979349 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:18.979323    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/1.log"
Jun 03 23:28:18.979729 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:18.979718    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/0.log"
Jun 03 23:28:18.979777 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:18.979750    2577 generic.go:356] "Generic (PLEG): container finished" podID="397370f1-b682-48ca-b643-f447f300c2a5" containerID="475f6db0f1d0b043d46da2358e2799b91d40221b6f2cd82283d79a96853868c0" exitCode=255
Jun 03 23:28:18.979825 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:18.979810    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l" event={"ID":"397370f1-b682-48ca-b643-f447f300c2a5","Type":"ContainerDied","Data":"475f6db0f1d0b043d46da2358e2799b91d40221b6f2cd82283d79a96853868c0"}
Jun 03 23:28:18.979877 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:18.979867    2577 scope.go:117] "RemoveContainer" containerID="b67f93dad7b43928ddb82d3c268e8ac4f32f84df7cabc1e1f4ba8eea0fe2d9dd"
Jun 03 23:28:18.980043 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:18.980027    2577 scope.go:117] "RemoveContainer" containerID="475f6db0f1d0b043d46da2358e2799b91d40221b6f2cd82283d79a96853868c0"
Jun 03 23:28:18.980235 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:18.980199    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85bd8fd97d-tsn6l_openshift-console-operator(397370f1-b682-48ca-b643-f447f300c2a5)\"" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l" podUID="397370f1-b682-48ca-b643-f447f300c2a5"
Jun 03 23:28:19.984417 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:19.984387    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/1.log"
Jun 03 23:28:19.984804 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:19.984702    2577 scope.go:117] "RemoveContainer" containerID="475f6db0f1d0b043d46da2358e2799b91d40221b6f2cd82283d79a96853868c0"
Jun 03 23:28:19.984855 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:19.984836    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85bd8fd97d-tsn6l_openshift-console-operator(397370f1-b682-48ca-b643-f447f300c2a5)\"" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l" podUID="397370f1-b682-48ca-b643-f447f300c2a5"
Jun 03 23:28:21.746093 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:21.746065    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n99j6_e42566fd-c3a2-48b7-8d03-a172d908aeff/dns-node-resolver/0.log"
Jun 03 23:28:22.263992 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.263960    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-rsmj7"]
Jun 03 23:28:22.266730 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.266713    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-rsmj7"
Jun 03 23:28:22.268762 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.268735    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jun 03 23:28:22.269617 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.269590    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jun 03 23:28:22.269720 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.269616    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jun 03 23:28:22.269720 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.269684    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-r8scv\""
Jun 03 23:28:22.269720 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.269690    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jun 03 23:28:22.281510 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.281486    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-rsmj7"]
Jun 03 23:28:22.343328 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.343289    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6bc5112d-47b2-450f-8995-cf497ca3858d-signing-key\") pod \"service-ca-7d74d578b8-rsmj7\" (UID: \"6bc5112d-47b2-450f-8995-cf497ca3858d\") " pod="openshift-service-ca/service-ca-7d74d578b8-rsmj7"
Jun 03 23:28:22.343482 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.343351    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6bc5112d-47b2-450f-8995-cf497ca3858d-signing-cabundle\") pod \"service-ca-7d74d578b8-rsmj7\" (UID: \"6bc5112d-47b2-450f-8995-cf497ca3858d\") " pod="openshift-service-ca/service-ca-7d74d578b8-rsmj7"
Jun 03 23:28:22.343482 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.343386    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfc8v\" (UniqueName: \"kubernetes.io/projected/6bc5112d-47b2-450f-8995-cf497ca3858d-kube-api-access-xfc8v\") pod \"service-ca-7d74d578b8-rsmj7\" (UID: \"6bc5112d-47b2-450f-8995-cf497ca3858d\") " pod="openshift-service-ca/service-ca-7d74d578b8-rsmj7"
Jun 03 23:28:22.443996 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.443963    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6bc5112d-47b2-450f-8995-cf497ca3858d-signing-key\") pod \"service-ca-7d74d578b8-rsmj7\" (UID: \"6bc5112d-47b2-450f-8995-cf497ca3858d\") " pod="openshift-service-ca/service-ca-7d74d578b8-rsmj7"
Jun 03 23:28:22.444140 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.444013    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6bc5112d-47b2-450f-8995-cf497ca3858d-signing-cabundle\") pod \"service-ca-7d74d578b8-rsmj7\" (UID: \"6bc5112d-47b2-450f-8995-cf497ca3858d\") " pod="openshift-service-ca/service-ca-7d74d578b8-rsmj7"
Jun 03 23:28:22.444140 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.444031    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfc8v\" (UniqueName: \"kubernetes.io/projected/6bc5112d-47b2-450f-8995-cf497ca3858d-kube-api-access-xfc8v\") pod \"service-ca-7d74d578b8-rsmj7\" (UID: \"6bc5112d-47b2-450f-8995-cf497ca3858d\") " pod="openshift-service-ca/service-ca-7d74d578b8-rsmj7"
Jun 03 23:28:22.444587 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.444567    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6bc5112d-47b2-450f-8995-cf497ca3858d-signing-cabundle\") pod \"service-ca-7d74d578b8-rsmj7\" (UID: \"6bc5112d-47b2-450f-8995-cf497ca3858d\") " pod="openshift-service-ca/service-ca-7d74d578b8-rsmj7"
Jun 03 23:28:22.446415 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.446395    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6bc5112d-47b2-450f-8995-cf497ca3858d-signing-key\") pod \"service-ca-7d74d578b8-rsmj7\" (UID: \"6bc5112d-47b2-450f-8995-cf497ca3858d\") " pod="openshift-service-ca/service-ca-7d74d578b8-rsmj7"
Jun 03 23:28:22.453248 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.453197    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfc8v\" (UniqueName: \"kubernetes.io/projected/6bc5112d-47b2-450f-8995-cf497ca3858d-kube-api-access-xfc8v\") pod \"service-ca-7d74d578b8-rsmj7\" (UID: \"6bc5112d-47b2-450f-8995-cf497ca3858d\") " pod="openshift-service-ca/service-ca-7d74d578b8-rsmj7"
Jun 03 23:28:22.545194 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.545112    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:22.545194 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.545148    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:22.545436 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:22.545324    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jun 03 23:28:22.545436 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:22.545324    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle podName:98372747-9a9e-4a1e-ad2f-d274d1db6889 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:30.545301051 +0000 UTC m=+232.707979118 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle") pod "router-default-55b9b48c7c-jw6cm" (UID: "98372747-9a9e-4a1e-ad2f-d274d1db6889") : configmap references non-existent config key: service-ca.crt
Jun 03 23:28:22.545436 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:22.545395    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs podName:98372747-9a9e-4a1e-ad2f-d274d1db6889 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:30.54538185 +0000 UTC m=+232.708059909 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs") pod "router-default-55b9b48c7c-jw6cm" (UID: "98372747-9a9e-4a1e-ad2f-d274d1db6889") : secret "router-metrics-certs-default" not found
Jun 03 23:28:22.545834 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.545817    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6n76l_b70ba3a5-03df-4847-9f88-12c8ae56099c/node-ca/0.log"
Jun 03 23:28:22.575584 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.575564    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-rsmj7"
Jun 03 23:28:22.646179 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.646151    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-57769895dd-bhxbc\" (UID: \"6125ef6f-2e47-4f8c-9668-274bdffe1ca2\") " pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:22.646344 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:22.646321    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 03 23:28:22.646406 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.646353    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:22.646406 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:22.646388    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls podName:6125ef6f-2e47-4f8c-9668-274bdffe1ca2 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:30.646367484 +0000 UTC m=+232.809045541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-57769895dd-bhxbc" (UID: "6125ef6f-2e47-4f8c-9668-274bdffe1ca2") : secret "cluster-monitoring-operator-tls" not found
Jun 03 23:28:22.646534 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:22.646452    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jun 03 23:28:22.646534 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:22.646469    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78d9cb695b-4hdm9: secret "image-registry-tls" not found
Jun 03 23:28:22.646610 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:22.646546    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls podName:c9b06e42-99fa-47e2-847e-47b427f82c56 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:30.646529158 +0000 UTC m=+232.809207228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls") pod "image-registry-78d9cb695b-4hdm9" (UID: "c9b06e42-99fa-47e2-847e-47b427f82c56") : secret "image-registry-tls" not found
Jun 03 23:28:22.692848 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.692820    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-rsmj7"]
Jun 03 23:28:22.695190 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:28:22.695166    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bc5112d_47b2_450f_8995_cf497ca3858d.slice/crio-d6b5e7d5a2d3cb3efab412a34e4209cd3f54d4baadfe17edefaf7878f4a31cdf WatchSource:0}: Error finding container d6b5e7d5a2d3cb3efab412a34e4209cd3f54d4baadfe17edefaf7878f4a31cdf: Status 404 returned error can't find the container with id d6b5e7d5a2d3cb3efab412a34e4209cd3f54d4baadfe17edefaf7878f4a31cdf
Jun 03 23:28:22.995076 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:22.995037    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-rsmj7" event={"ID":"6bc5112d-47b2-450f-8995-cf497ca3858d","Type":"ContainerStarted","Data":"d6b5e7d5a2d3cb3efab412a34e4209cd3f54d4baadfe17edefaf7878f4a31cdf"}
Jun 03 23:28:23.347473 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:23.347443    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-28f9z_201466a6-58e5-49a6-869f-2161d2c8b708/serve-healthcheck-canary/0.log"
Jun 03 23:28:25.002523 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:25.002424    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-rsmj7" event={"ID":"6bc5112d-47b2-450f-8995-cf497ca3858d","Type":"ContainerStarted","Data":"ae6c26f6dbf15301e70fba6b7e44536eb0abd91dfe352210626965f9836235af"}
Jun 03 23:28:25.321199 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:25.321157    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:25.321359 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:25.321241    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:25.321581 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:25.321568    2577 scope.go:117] "RemoveContainer" containerID="475f6db0f1d0b043d46da2358e2799b91d40221b6f2cd82283d79a96853868c0"
Jun 03 23:28:25.321731 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:25.321714    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85bd8fd97d-tsn6l_openshift-console-operator(397370f1-b682-48ca-b643-f447f300c2a5)\"" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l" podUID="397370f1-b682-48ca-b643-f447f300c2a5"
Jun 03 23:28:30.611291 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:30.611247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:30.611291 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:30.611294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:30.611728 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:30.611421    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle podName:98372747-9a9e-4a1e-ad2f-d274d1db6889 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:46.611403505 +0000 UTC m=+248.774081557 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle") pod "router-default-55b9b48c7c-jw6cm" (UID: "98372747-9a9e-4a1e-ad2f-d274d1db6889") : configmap references non-existent config key: service-ca.crt
Jun 03 23:28:30.613691 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:30.613667    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98372747-9a9e-4a1e-ad2f-d274d1db6889-metrics-certs\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:30.712593 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:30.712563    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:30.712753 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:30.712621    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-57769895dd-bhxbc\" (UID: \"6125ef6f-2e47-4f8c-9668-274bdffe1ca2\") " pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:30.712824 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:30.712751    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jun 03 23:28:30.712869 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:30.712834    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls podName:6125ef6f-2e47-4f8c-9668-274bdffe1ca2 nodeName:}" failed. No retries permitted until 2026-06-03 23:28:46.712810139 +0000 UTC m=+248.875488205 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-57769895dd-bhxbc" (UID: "6125ef6f-2e47-4f8c-9668-274bdffe1ca2") : secret "cluster-monitoring-operator-tls" not found
Jun 03 23:28:30.714975 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:30.714949    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls\") pod \"image-registry-78d9cb695b-4hdm9\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") " pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:30.839981 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:30.839946    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:30.968009 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:30.967950    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7d74d578b8-rsmj7" podStartSLOduration=7.050271895 podStartE2EDuration="8.967929396s" totalImagesPullingTime="1.917657501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:28:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:28:25.017862018 +0000 UTC m=+227.180540083" watchObservedRunningTime="2026-06-03 23:28:30.967929396 +0000 UTC m=+233.130607471"
Jun 03 23:28:30.968457 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:30.968437    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78d9cb695b-4hdm9"]
Jun 03 23:28:30.971476 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:28:30.971453    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9b06e42_99fa_47e2_847e_47b427f82c56.slice/crio-9a7dca6b167a55777d2d6e80fc3c49ef00b66dddd55c96a907a9735c043b8277 WatchSource:0}: Error finding container 9a7dca6b167a55777d2d6e80fc3c49ef00b66dddd55c96a907a9735c043b8277: Status 404 returned error can't find the container with id 9a7dca6b167a55777d2d6e80fc3c49ef00b66dddd55c96a907a9735c043b8277
Jun 03 23:28:31.020538 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:31.020506    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9" event={"ID":"c9b06e42-99fa-47e2-847e-47b427f82c56","Type":"ContainerStarted","Data":"9a7dca6b167a55777d2d6e80fc3c49ef00b66dddd55c96a907a9735c043b8277"}
Jun 03 23:28:32.024848 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:32.024809    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9" event={"ID":"c9b06e42-99fa-47e2-847e-47b427f82c56","Type":"ContainerStarted","Data":"b6c87e819549784e3646dad0334d25efb1b981f1cee15bb1cb9110fc467a9b1a"}
Jun 03 23:28:32.025243 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:32.024932    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:28:32.046435 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:32.046381    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9" podStartSLOduration=18.046366026 podStartE2EDuration="18.046366026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:28:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:28:32.045259822 +0000 UTC m=+234.207937896" watchObservedRunningTime="2026-06-03 23:28:32.046366026 +0000 UTC m=+234.209044130"
Jun 03 23:28:38.274693 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:38.274659    2577 scope.go:117] "RemoveContainer" containerID="475f6db0f1d0b043d46da2358e2799b91d40221b6f2cd82283d79a96853868c0"
Jun 03 23:28:39.047219 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:39.047176    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 03 23:28:39.047628 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:39.047613    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/1.log"
Jun 03 23:28:39.047688 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:39.047648    2577 generic.go:356] "Generic (PLEG): container finished" podID="397370f1-b682-48ca-b643-f447f300c2a5" containerID="52af4cfe51125380c67e17418fb859a216ef7bc1a16ff938fced93510c401896" exitCode=255
Jun 03 23:28:39.047727 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:39.047699    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l" event={"ID":"397370f1-b682-48ca-b643-f447f300c2a5","Type":"ContainerDied","Data":"52af4cfe51125380c67e17418fb859a216ef7bc1a16ff938fced93510c401896"}
Jun 03 23:28:39.047761 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:39.047747    2577 scope.go:117] "RemoveContainer" containerID="475f6db0f1d0b043d46da2358e2799b91d40221b6f2cd82283d79a96853868c0"
Jun 03 23:28:39.048128 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:39.048108    2577 scope.go:117] "RemoveContainer" containerID="52af4cfe51125380c67e17418fb859a216ef7bc1a16ff938fced93510c401896"
Jun 03 23:28:39.048325 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:39.048306    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85bd8fd97d-tsn6l_openshift-console-operator(397370f1-b682-48ca-b643-f447f300c2a5)\"" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l" podUID="397370f1-b682-48ca-b643-f447f300c2a5"
Jun 03 23:28:40.052742 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:40.052714    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 03 23:28:44.983024 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:44.982984    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wwfjd"]
Jun 03 23:28:44.990388 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:44.990359    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:44.993706 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:44.993682    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jun 03 23:28:44.993836 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:44.993717    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jun 03 23:28:44.993836 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:44.993684    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jun 03 23:28:44.993836 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:44.993805    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h745f\""
Jun 03 23:28:44.994099 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:44.994075    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jun 03 23:28:44.994174 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:44.994129    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78d9cb695b-4hdm9"]
Jun 03 23:28:45.002151 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.002130    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wwfjd"]
Jun 03 23:28:45.131370 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.131336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ae38cd56-f7e1-44da-84b7-90f45b54b423-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wwfjd\" (UID: \"ae38cd56-f7e1-44da-84b7-90f45b54b423\") " pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:45.131535 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.131406    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ae38cd56-f7e1-44da-84b7-90f45b54b423-crio-socket\") pod \"insights-runtime-extractor-wwfjd\" (UID: \"ae38cd56-f7e1-44da-84b7-90f45b54b423\") " pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:45.131535 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.131463    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ltn7\" (UniqueName: \"kubernetes.io/projected/ae38cd56-f7e1-44da-84b7-90f45b54b423-kube-api-access-6ltn7\") pod \"insights-runtime-extractor-wwfjd\" (UID: \"ae38cd56-f7e1-44da-84b7-90f45b54b423\") " pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:45.131535 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.131498    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae38cd56-f7e1-44da-84b7-90f45b54b423-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wwfjd\" (UID: \"ae38cd56-f7e1-44da-84b7-90f45b54b423\") " pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:45.131535 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.131524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae38cd56-f7e1-44da-84b7-90f45b54b423-data-volume\") pod \"insights-runtime-extractor-wwfjd\" (UID: \"ae38cd56-f7e1-44da-84b7-90f45b54b423\") " pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:45.232418 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.232382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae38cd56-f7e1-44da-84b7-90f45b54b423-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wwfjd\" (UID: \"ae38cd56-f7e1-44da-84b7-90f45b54b423\") " pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:45.232418 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.232424    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae38cd56-f7e1-44da-84b7-90f45b54b423-data-volume\") pod \"insights-runtime-extractor-wwfjd\" (UID: \"ae38cd56-f7e1-44da-84b7-90f45b54b423\") " pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:45.232640 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.232484    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ae38cd56-f7e1-44da-84b7-90f45b54b423-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wwfjd\" (UID: \"ae38cd56-f7e1-44da-84b7-90f45b54b423\") " pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:45.232640 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.232553    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ae38cd56-f7e1-44da-84b7-90f45b54b423-crio-socket\") pod \"insights-runtime-extractor-wwfjd\" (UID: \"ae38cd56-f7e1-44da-84b7-90f45b54b423\") " pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:45.232640 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.232586    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ltn7\" (UniqueName: \"kubernetes.io/projected/ae38cd56-f7e1-44da-84b7-90f45b54b423-kube-api-access-6ltn7\") pod \"insights-runtime-extractor-wwfjd\" (UID: \"ae38cd56-f7e1-44da-84b7-90f45b54b423\") " pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:45.232783 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.232698    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ae38cd56-f7e1-44da-84b7-90f45b54b423-crio-socket\") pod \"insights-runtime-extractor-wwfjd\" (UID: \"ae38cd56-f7e1-44da-84b7-90f45b54b423\") " pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:45.232861 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.232829    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae38cd56-f7e1-44da-84b7-90f45b54b423-data-volume\") pod \"insights-runtime-extractor-wwfjd\" (UID: \"ae38cd56-f7e1-44da-84b7-90f45b54b423\") " pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:45.233201 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.233149    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ae38cd56-f7e1-44da-84b7-90f45b54b423-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wwfjd\" (UID: \"ae38cd56-f7e1-44da-84b7-90f45b54b423\") " pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:45.234650 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.234633    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae38cd56-f7e1-44da-84b7-90f45b54b423-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wwfjd\" (UID: \"ae38cd56-f7e1-44da-84b7-90f45b54b423\") " pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:45.242776 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.242758    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ltn7\" (UniqueName: \"kubernetes.io/projected/ae38cd56-f7e1-44da-84b7-90f45b54b423-kube-api-access-6ltn7\") pod \"insights-runtime-extractor-wwfjd\" (UID: \"ae38cd56-f7e1-44da-84b7-90f45b54b423\") " pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:45.300704 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.300669    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wwfjd"
Jun 03 23:28:45.320850 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.320815    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:45.320960 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.320872    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:28:45.321201 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.321188    2577 scope.go:117] "RemoveContainer" containerID="52af4cfe51125380c67e17418fb859a216ef7bc1a16ff938fced93510c401896"
Jun 03 23:28:45.321420 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:45.321398    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85bd8fd97d-tsn6l_openshift-console-operator(397370f1-b682-48ca-b643-f447f300c2a5)\"" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l" podUID="397370f1-b682-48ca-b643-f447f300c2a5"
Jun 03 23:28:45.431758 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:45.431715    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wwfjd"]
Jun 03 23:28:45.434222 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:28:45.434170    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae38cd56_f7e1_44da_84b7_90f45b54b423.slice/crio-b5cf2a98dbb6f09e2b484258373a17a29866070e55a48f42e4b59a7dc873eb5d WatchSource:0}: Error finding container b5cf2a98dbb6f09e2b484258373a17a29866070e55a48f42e4b59a7dc873eb5d: Status 404 returned error can't find the container with id b5cf2a98dbb6f09e2b484258373a17a29866070e55a48f42e4b59a7dc873eb5d
Jun 03 23:28:46.070978 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:46.070945    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wwfjd" event={"ID":"ae38cd56-f7e1-44da-84b7-90f45b54b423","Type":"ContainerStarted","Data":"d7a9521709249ec2616d871413596fd380183b9e7080c5c384c2ca1a2279f0c1"}
Jun 03 23:28:46.071497 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:46.071005    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wwfjd" event={"ID":"ae38cd56-f7e1-44da-84b7-90f45b54b423","Type":"ContainerStarted","Data":"b5cf2a98dbb6f09e2b484258373a17a29866070e55a48f42e4b59a7dc873eb5d"}
Jun 03 23:28:46.645355 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:46.645315    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:46.645912 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:46.645892    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98372747-9a9e-4a1e-ad2f-d274d1db6889-service-ca-bundle\") pod \"router-default-55b9b48c7c-jw6cm\" (UID: \"98372747-9a9e-4a1e-ad2f-d274d1db6889\") " pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:46.745995 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:46.745954    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-57769895dd-bhxbc\" (UID: \"6125ef6f-2e47-4f8c-9668-274bdffe1ca2\") " pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:46.748762 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:46.748731    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6125ef6f-2e47-4f8c-9668-274bdffe1ca2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-57769895dd-bhxbc\" (UID: \"6125ef6f-2e47-4f8c-9668-274bdffe1ca2\") " pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:46.925729 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:46.925643    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-pvpbs\""
Jun 03 23:28:46.933602 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:46.933586    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:47.035982 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:47.035939    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-69w9s\""
Jun 03 23:28:47.044325 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:47.044260    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"
Jun 03 23:28:47.078015 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:47.077973    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wwfjd" event={"ID":"ae38cd56-f7e1-44da-84b7-90f45b54b423","Type":"ContainerStarted","Data":"96cf07e201fff55d6a7475ebe82e2c9b19ec9b3b8b8814e719ec5c8fe4a132c6"}
Jun 03 23:28:47.089289 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:47.089262    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-55b9b48c7c-jw6cm"]
Jun 03 23:28:47.091977 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:28:47.091948    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98372747_9a9e_4a1e_ad2f_d274d1db6889.slice/crio-d34560d7fe3fa87b9ac4e34be60bd605e258c73350a2a1ef72992751a96fbf11 WatchSource:0}: Error finding container d34560d7fe3fa87b9ac4e34be60bd605e258c73350a2a1ef72992751a96fbf11: Status 404 returned error can't find the container with id d34560d7fe3fa87b9ac4e34be60bd605e258c73350a2a1ef72992751a96fbf11
Jun 03 23:28:47.215942 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:47.215880    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc"]
Jun 03 23:28:47.218841 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:28:47.218585    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6125ef6f_2e47_4f8c_9668_274bdffe1ca2.slice/crio-0096fe0ad4efaf9143a5964761de70aba1cf89c724b8eef4a083b336a452eb15 WatchSource:0}: Error finding container 0096fe0ad4efaf9143a5964761de70aba1cf89c724b8eef4a083b336a452eb15: Status 404 returned error can't find the container with id 0096fe0ad4efaf9143a5964761de70aba1cf89c724b8eef4a083b336a452eb15
Jun 03 23:28:48.084112 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:48.084078    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wwfjd" event={"ID":"ae38cd56-f7e1-44da-84b7-90f45b54b423","Type":"ContainerStarted","Data":"1fe5803c68b6b7c3584e667cf76d32de96817bfff9565522f0fa99a8bba71d7c"}
Jun 03 23:28:48.085415 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:48.085389    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc" event={"ID":"6125ef6f-2e47-4f8c-9668-274bdffe1ca2","Type":"ContainerStarted","Data":"0096fe0ad4efaf9143a5964761de70aba1cf89c724b8eef4a083b336a452eb15"}
Jun 03 23:28:48.086837 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:48.086805    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-55b9b48c7c-jw6cm" event={"ID":"98372747-9a9e-4a1e-ad2f-d274d1db6889","Type":"ContainerStarted","Data":"5f7779278f91ce562e86d995045eaf3c4ca0dd39eee1fa94df5f512071599083"}
Jun 03 23:28:48.086947 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:48.086854    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-55b9b48c7c-jw6cm" event={"ID":"98372747-9a9e-4a1e-ad2f-d274d1db6889","Type":"ContainerStarted","Data":"d34560d7fe3fa87b9ac4e34be60bd605e258c73350a2a1ef72992751a96fbf11"}
Jun 03 23:28:48.103436 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:48.103389    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wwfjd" podStartSLOduration=1.879541242 podStartE2EDuration="4.103374389s" totalImagesPullingTime="2.223833147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-03 23:28:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:28:48.102142822 +0000 UTC m=+250.264820909" watchObservedRunningTime="2026-06-03 23:28:48.103374389 +0000 UTC m=+250.266052466"
Jun 03 23:28:48.122055 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:48.121919    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-55b9b48c7c-jw6cm" podStartSLOduration=34.121903102 podStartE2EDuration="34.121903102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:28:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:28:48.121740687 +0000 UTC m=+250.284418762" watchObservedRunningTime="2026-06-03 23:28:48.121903102 +0000 UTC m=+250.284581177"
Jun 03 23:28:48.934730 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:48.934684    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:48.937344 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:48.937320    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:49.064581 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:49.064547    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs\") pod \"network-metrics-daemon-ssbpc\" (UID: \"81ee1ff9-1018-4eaf-b246-18b5f7a95f78\") " pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:28:49.067124 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:49.067089    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81ee1ff9-1018-4eaf-b246-18b5f7a95f78-metrics-certs\") pod \"network-metrics-daemon-ssbpc\" (UID: \"81ee1ff9-1018-4eaf-b246-18b5f7a95f78\") " pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:28:49.076485 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:49.076465    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8mqss\""
Jun 03 23:28:49.084405 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:49.084387    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ssbpc"
Jun 03 23:28:49.090189 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:49.090172    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:49.091239 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:49.091218    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-55b9b48c7c-jw6cm"
Jun 03 23:28:49.226439 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:49.226416    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ssbpc"]
Jun 03 23:28:49.228794 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:28:49.228767    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81ee1ff9_1018_4eaf_b246_18b5f7a95f78.slice/crio-5d0099a12bee3856d1d99c4e09f0f0bea1a64d58d7e744a86955d3bb5d70efd8 WatchSource:0}: Error finding container 5d0099a12bee3856d1d99c4e09f0f0bea1a64d58d7e744a86955d3bb5d70efd8: Status 404 returned error can't find the container with id 5d0099a12bee3856d1d99c4e09f0f0bea1a64d58d7e744a86955d3bb5d70efd8
Jun 03 23:28:50.094764 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:50.094562    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc" event={"ID":"6125ef6f-2e47-4f8c-9668-274bdffe1ca2","Type":"ContainerStarted","Data":"6faf3f804708f863c879832bfd008ab0ba25fe7a67213aebabeeba094a84a6a5"}
Jun 03 23:28:50.095993 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:50.095967    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ssbpc" event={"ID":"81ee1ff9-1018-4eaf-b246-18b5f7a95f78","Type":"ContainerStarted","Data":"5d0099a12bee3856d1d99c4e09f0f0bea1a64d58d7e744a86955d3bb5d70efd8"}
Jun 03 23:28:50.116611 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:50.116562    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-57769895dd-bhxbc" podStartSLOduration=34.208998982 podStartE2EDuration="36.116546846s" totalImagesPullingTime="1.907547864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:28:50.115535122 +0000 UTC m=+252.278213197" watchObservedRunningTime="2026-06-03 23:28:50.116546846 +0000 UTC m=+252.279224899"
Jun 03 23:28:51.101847 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:51.101757    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ssbpc" event={"ID":"81ee1ff9-1018-4eaf-b246-18b5f7a95f78","Type":"ContainerStarted","Data":"00e19337065bf6429a371dc81920b05686e810d6345e4a3b3f6fbd4470dd08ab"}
Jun 03 23:28:51.101847 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:51.101811    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ssbpc" event={"ID":"81ee1ff9-1018-4eaf-b246-18b5f7a95f78","Type":"ContainerStarted","Data":"21536b4aaf74dcbc69293edb6eb5bbad4cf3ab82145d2cbf329706605574fc7d"}
Jun 03 23:28:51.121054 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:51.120614    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ssbpc" podStartSLOduration=252.315017215 podStartE2EDuration="4m13.120595086s" totalImagesPullingTime="805.577871ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:28:51.119787903 +0000 UTC m=+253.282465977" watchObservedRunningTime="2026-06-03 23:28:51.120595086 +0000 UTC m=+253.283273161"
Jun 03 23:28:53.758742 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:53.758701    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"]
Jun 03 23:28:53.768084 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:53.768055    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"
Jun 03 23:28:53.770487 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:53.770452    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"]
Jun 03 23:28:53.771167 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:53.770994    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-blpb6\""
Jun 03 23:28:53.771167 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:53.771063    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jun 03 23:28:53.771595 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:53.771575    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jun 03 23:28:53.771827 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:53.771813    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jun 03 23:28:53.902574 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:53.902535    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/800518f3-21a0-4933-9699-be4f6b7dfeb9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-vfmd8\" (UID: \"800518f3-21a0-4933-9699-be4f6b7dfeb9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"
Jun 03 23:28:53.902771 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:53.902605    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzvbn\" (UniqueName: \"kubernetes.io/projected/800518f3-21a0-4933-9699-be4f6b7dfeb9-kube-api-access-xzvbn\") pod \"prometheus-operator-7f7d445d84-vfmd8\" (UID: \"800518f3-21a0-4933-9699-be4f6b7dfeb9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"
Jun 03 23:28:53.902771 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:53.902639    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/800518f3-21a0-4933-9699-be4f6b7dfeb9-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-vfmd8\" (UID: \"800518f3-21a0-4933-9699-be4f6b7dfeb9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"
Jun 03 23:28:53.902771 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:53.902693    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/800518f3-21a0-4933-9699-be4f6b7dfeb9-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-vfmd8\" (UID: \"800518f3-21a0-4933-9699-be4f6b7dfeb9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"
Jun 03 23:28:54.003456 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:54.003418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/800518f3-21a0-4933-9699-be4f6b7dfeb9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-vfmd8\" (UID: \"800518f3-21a0-4933-9699-be4f6b7dfeb9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"
Jun 03 23:28:54.003642 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:54.003485    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzvbn\" (UniqueName: \"kubernetes.io/projected/800518f3-21a0-4933-9699-be4f6b7dfeb9-kube-api-access-xzvbn\") pod \"prometheus-operator-7f7d445d84-vfmd8\" (UID: \"800518f3-21a0-4933-9699-be4f6b7dfeb9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"
Jun 03 23:28:54.003642 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:54.003519    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/800518f3-21a0-4933-9699-be4f6b7dfeb9-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-vfmd8\" (UID: \"800518f3-21a0-4933-9699-be4f6b7dfeb9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"
Jun 03 23:28:54.003642 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:54.003577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/800518f3-21a0-4933-9699-be4f6b7dfeb9-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-vfmd8\" (UID: \"800518f3-21a0-4933-9699-be4f6b7dfeb9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"
Jun 03 23:28:54.004165 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:54.004144    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/800518f3-21a0-4933-9699-be4f6b7dfeb9-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-vfmd8\" (UID: \"800518f3-21a0-4933-9699-be4f6b7dfeb9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"
Jun 03 23:28:54.005857 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:54.005834    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/800518f3-21a0-4933-9699-be4f6b7dfeb9-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-vfmd8\" (UID: \"800518f3-21a0-4933-9699-be4f6b7dfeb9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"
Jun 03 23:28:54.005857 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:54.005849    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/800518f3-21a0-4933-9699-be4f6b7dfeb9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-vfmd8\" (UID: \"800518f3-21a0-4933-9699-be4f6b7dfeb9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"
Jun 03 23:28:54.012510 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:54.012456    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzvbn\" (UniqueName: \"kubernetes.io/projected/800518f3-21a0-4933-9699-be4f6b7dfeb9-kube-api-access-xzvbn\") pod \"prometheus-operator-7f7d445d84-vfmd8\" (UID: \"800518f3-21a0-4933-9699-be4f6b7dfeb9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"
Jun 03 23:28:54.078250 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:54.078219    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"
Jun 03 23:28:54.201139 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:54.201112    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8"]
Jun 03 23:28:54.203962 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:28:54.203938    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod800518f3_21a0_4933_9699_be4f6b7dfeb9.slice/crio-118c74b7d7f4f7646be0b363f672c87a3cf4678cf3de58c14707bb629deda351 WatchSource:0}: Error finding container 118c74b7d7f4f7646be0b363f672c87a3cf4678cf3de58c14707bb629deda351: Status 404 returned error can't find the container with id 118c74b7d7f4f7646be0b363f672c87a3cf4678cf3de58c14707bb629deda351
Jun 03 23:28:55.001972 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:55.001920    2577 patch_prober.go:28] interesting pod/image-registry-78d9cb695b-4hdm9 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jun 03 23:28:55.002457 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:55.001987    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9" podUID="c9b06e42-99fa-47e2-847e-47b427f82c56" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 03 23:28:55.116254 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:55.116198    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8" event={"ID":"800518f3-21a0-4933-9699-be4f6b7dfeb9","Type":"ContainerStarted","Data":"118c74b7d7f4f7646be0b363f672c87a3cf4678cf3de58c14707bb629deda351"}
Jun 03 23:28:56.121298 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:56.121262    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8" event={"ID":"800518f3-21a0-4933-9699-be4f6b7dfeb9","Type":"ContainerStarted","Data":"ebe50ac1e50f6ace843d336c6d817b00831c68ee267f7ec6c8bea3d7fba72986"}
Jun 03 23:28:56.121681 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:56.121314    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8" event={"ID":"800518f3-21a0-4933-9699-be4f6b7dfeb9","Type":"ContainerStarted","Data":"5bbfd1860017d69e9721a9b4f756d6d46d294fba79ef49c4f2dec209092ae551"}
Jun 03 23:28:56.139031 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:56.138970    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7f7d445d84-vfmd8" podStartSLOduration=1.83123568 podStartE2EDuration="3.138949984s" totalImagesPullingTime="1.307714304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:28:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:28:56.138001748 +0000 UTC m=+258.300679821" watchObservedRunningTime="2026-06-03 23:28:56.138949984 +0000 UTC m=+258.301628060"
Jun 03 23:28:58.144513 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.144482    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7zqrp"]
Jun 03 23:28:58.146710 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.146693    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.148879 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.148857    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5kp99\""
Jun 03 23:28:58.148996 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.148958    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jun 03 23:28:58.149156 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.149136    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jun 03 23:28:58.149240 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.149218    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jun 03 23:28:58.238291 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.238256    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8fb4d18f-a068-4cec-8181-bc88350b50c8-sys\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.238291 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.238290    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8fb4d18f-a068-4cec-8181-bc88350b50c8-node-exporter-wtmp\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.238489 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.238314    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8fb4d18f-a068-4cec-8181-bc88350b50c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.238489 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.238388    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8fb4d18f-a068-4cec-8181-bc88350b50c8-node-exporter-textfile\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.238489 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.238439    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8fb4d18f-a068-4cec-8181-bc88350b50c8-root\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.238489 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.238470    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8fb4d18f-a068-4cec-8181-bc88350b50c8-node-exporter-tls\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.238608 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.238495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fb4d18f-a068-4cec-8181-bc88350b50c8-metrics-client-ca\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.238608 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.238522    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvzjq\" (UniqueName: \"kubernetes.io/projected/8fb4d18f-a068-4cec-8181-bc88350b50c8-kube-api-access-nvzjq\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.238608 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.238571    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8fb4d18f-a068-4cec-8181-bc88350b50c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.274897 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.274875    2577 scope.go:117] "RemoveContainer" containerID="52af4cfe51125380c67e17418fb859a216ef7bc1a16ff938fced93510c401896"
Jun 03 23:28:58.275046 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:28:58.275028    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85bd8fd97d-tsn6l_openshift-console-operator(397370f1-b682-48ca-b643-f447f300c2a5)\"" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l" podUID="397370f1-b682-48ca-b643-f447f300c2a5"
Jun 03 23:28:58.339517 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.339481    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8fb4d18f-a068-4cec-8181-bc88350b50c8-root\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.339517 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.339518    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8fb4d18f-a068-4cec-8181-bc88350b50c8-node-exporter-tls\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.339746 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.339536    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fb4d18f-a068-4cec-8181-bc88350b50c8-metrics-client-ca\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.339746 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.339561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvzjq\" (UniqueName: \"kubernetes.io/projected/8fb4d18f-a068-4cec-8181-bc88350b50c8-kube-api-access-nvzjq\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.339746 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.339599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8fb4d18f-a068-4cec-8181-bc88350b50c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.339746 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.339604    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8fb4d18f-a068-4cec-8181-bc88350b50c8-root\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.339746 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.339719    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8fb4d18f-a068-4cec-8181-bc88350b50c8-sys\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.339978 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.339750    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8fb4d18f-a068-4cec-8181-bc88350b50c8-node-exporter-wtmp\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.339978 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.339781    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8fb4d18f-a068-4cec-8181-bc88350b50c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.339978 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.339815    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8fb4d18f-a068-4cec-8181-bc88350b50c8-sys\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.339978 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.339823    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8fb4d18f-a068-4cec-8181-bc88350b50c8-node-exporter-textfile\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.339978 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.339945    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8fb4d18f-a068-4cec-8181-bc88350b50c8-node-exporter-wtmp\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.340195 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.340082    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8fb4d18f-a068-4cec-8181-bc88350b50c8-node-exporter-textfile\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.340266 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.340189    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fb4d18f-a068-4cec-8181-bc88350b50c8-metrics-client-ca\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.340343 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.340321    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8fb4d18f-a068-4cec-8181-bc88350b50c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.341972 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.341955    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8fb4d18f-a068-4cec-8181-bc88350b50c8-node-exporter-tls\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.342054 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.341977    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8fb4d18f-a068-4cec-8181-bc88350b50c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.349049 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.349028    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvzjq\" (UniqueName: \"kubernetes.io/projected/8fb4d18f-a068-4cec-8181-bc88350b50c8-kube-api-access-nvzjq\") pod \"node-exporter-7zqrp\" (UID: \"8fb4d18f-a068-4cec-8181-bc88350b50c8\") " pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.458375 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:58.458299    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7zqrp"
Jun 03 23:28:58.467686 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:28:58.467659    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fb4d18f_a068_4cec_8181_bc88350b50c8.slice/crio-b80aa59debf0b55f626609244268703cf2a7472807a11a335a8da1ab51d2454b WatchSource:0}: Error finding container b80aa59debf0b55f626609244268703cf2a7472807a11a335a8da1ab51d2454b: Status 404 returned error can't find the container with id b80aa59debf0b55f626609244268703cf2a7472807a11a335a8da1ab51d2454b
Jun 03 23:28:59.133899 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:28:59.133855    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7zqrp" event={"ID":"8fb4d18f-a068-4cec-8181-bc88350b50c8","Type":"ContainerStarted","Data":"b80aa59debf0b55f626609244268703cf2a7472807a11a335a8da1ab51d2454b"}
Jun 03 23:29:00.138559 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:00.138519    2577 generic.go:356] "Generic (PLEG): container finished" podID="8fb4d18f-a068-4cec-8181-bc88350b50c8" containerID="ba393c7520e523a98d8124f1f5252aa96183b9dd59e866674e6a2972de841c08" exitCode=0
Jun 03 23:29:00.139021 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:00.138581    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7zqrp" event={"ID":"8fb4d18f-a068-4cec-8181-bc88350b50c8","Type":"ContainerDied","Data":"ba393c7520e523a98d8124f1f5252aa96183b9dd59e866674e6a2972de841c08"}
Jun 03 23:29:01.144702 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:01.144668    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7zqrp" event={"ID":"8fb4d18f-a068-4cec-8181-bc88350b50c8","Type":"ContainerStarted","Data":"35abac5707aeb5c9d229f9f1ad7671a94fa8547540dd43635f1d2eb8efaf2999"}
Jun 03 23:29:01.145147 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:01.144721    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7zqrp" event={"ID":"8fb4d18f-a068-4cec-8181-bc88350b50c8","Type":"ContainerStarted","Data":"d464681ff6a7f8265406c77a142217d6a2b95c871ce84ecce990b1808dbaa1ca"}
Jun 03 23:29:01.167553 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:01.167506    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7zqrp" podStartSLOduration=2.251512112 podStartE2EDuration="3.167489987s" totalImagesPullingTime="849.27921ms" totalInitContainerRuntime="66.698665ms" isStatefulPod=true podCreationTimestamp="2026-06-03 23:28:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:29:01.166865415 +0000 UTC m=+263.329543490" watchObservedRunningTime="2026-06-03 23:29:01.167489987 +0000 UTC m=+263.330168055"
Jun 03 23:29:02.883533 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:02.883490    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-9b9fc658-m49nl"]
Jun 03 23:29:02.885967 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:02.885950    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-9b9fc658-m49nl"
Jun 03 23:29:02.888296 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:02.888267    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-z7m5q\""
Jun 03 23:29:02.888417 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:02.888271    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jun 03 23:29:02.896669 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:02.896645    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-9b9fc658-m49nl"]
Jun 03 23:29:03.078779 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:03.078743    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxx68\" (UniqueName: \"kubernetes.io/projected/8a6149e0-57e9-4741-99df-0198dcaafa03-kube-api-access-qxx68\") pod \"monitoring-plugin-9b9fc658-m49nl\" (UID: \"8a6149e0-57e9-4741-99df-0198dcaafa03\") " pod="openshift-monitoring/monitoring-plugin-9b9fc658-m49nl"
Jun 03 23:29:03.078779 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:03.078784    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8a6149e0-57e9-4741-99df-0198dcaafa03-monitoring-plugin-cert\") pod \"monitoring-plugin-9b9fc658-m49nl\" (UID: \"8a6149e0-57e9-4741-99df-0198dcaafa03\") " pod="openshift-monitoring/monitoring-plugin-9b9fc658-m49nl"
Jun 03 23:29:03.179844 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:03.179755    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxx68\" (UniqueName: \"kubernetes.io/projected/8a6149e0-57e9-4741-99df-0198dcaafa03-kube-api-access-qxx68\") pod \"monitoring-plugin-9b9fc658-m49nl\" (UID: \"8a6149e0-57e9-4741-99df-0198dcaafa03\") " pod="openshift-monitoring/monitoring-plugin-9b9fc658-m49nl"
Jun 03 23:29:03.179844 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:03.179794    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8a6149e0-57e9-4741-99df-0198dcaafa03-monitoring-plugin-cert\") pod \"monitoring-plugin-9b9fc658-m49nl\" (UID: \"8a6149e0-57e9-4741-99df-0198dcaafa03\") " pod="openshift-monitoring/monitoring-plugin-9b9fc658-m49nl"
Jun 03 23:29:03.182296 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:03.182272    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8a6149e0-57e9-4741-99df-0198dcaafa03-monitoring-plugin-cert\") pod \"monitoring-plugin-9b9fc658-m49nl\" (UID: \"8a6149e0-57e9-4741-99df-0198dcaafa03\") " pod="openshift-monitoring/monitoring-plugin-9b9fc658-m49nl"
Jun 03 23:29:03.198717 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:03.198684    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxx68\" (UniqueName: \"kubernetes.io/projected/8a6149e0-57e9-4741-99df-0198dcaafa03-kube-api-access-qxx68\") pod \"monitoring-plugin-9b9fc658-m49nl\" (UID: \"8a6149e0-57e9-4741-99df-0198dcaafa03\") " pod="openshift-monitoring/monitoring-plugin-9b9fc658-m49nl"
Jun 03 23:29:03.496380 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:03.496277    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-9b9fc658-m49nl"
Jun 03 23:29:03.622027 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:03.621988    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-9b9fc658-m49nl"]
Jun 03 23:29:03.627115 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:29:03.627085    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a6149e0_57e9_4741_99df_0198dcaafa03.slice/crio-08f6f5469ea38d8c234fb0a1c22b73c05befeb526851d3fcbddfa2eac9af0ca2 WatchSource:0}: Error finding container 08f6f5469ea38d8c234fb0a1c22b73c05befeb526851d3fcbddfa2eac9af0ca2: Status 404 returned error can't find the container with id 08f6f5469ea38d8c234fb0a1c22b73c05befeb526851d3fcbddfa2eac9af0ca2
Jun 03 23:29:04.160084 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:04.160048    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-9b9fc658-m49nl" event={"ID":"8a6149e0-57e9-4741-99df-0198dcaafa03","Type":"ContainerStarted","Data":"08f6f5469ea38d8c234fb0a1c22b73c05befeb526851d3fcbddfa2eac9af0ca2"}
Jun 03 23:29:04.999385 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:04.999355    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:29:06.168172 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:06.168136    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-9b9fc658-m49nl" event={"ID":"8a6149e0-57e9-4741-99df-0198dcaafa03","Type":"ContainerStarted","Data":"325fdae5d325d1bf2de3bba9cc41a4dfe92da98300ef898b5c3b641a75e6d3fa"}
Jun 03 23:29:06.168625 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:06.168322    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-9b9fc658-m49nl"
Jun 03 23:29:06.173521 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:06.173499    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-9b9fc658-m49nl"
Jun 03 23:29:06.190832 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:06.190781    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-9b9fc658-m49nl" podStartSLOduration=2.669030432 podStartE2EDuration="4.190766413s" totalImagesPullingTime="1.521735981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:29:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:29:06.190043024 +0000 UTC m=+268.352721097" watchObservedRunningTime="2026-06-03 23:29:06.190766413 +0000 UTC m=+268.353444487"
Jun 03 23:29:10.014657 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.014611    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9" podUID="c9b06e42-99fa-47e2-847e-47b427f82c56" containerName="registry" containerID="cri-o://b6c87e819549784e3646dad0334d25efb1b981f1cee15bb1cb9110fc467a9b1a" gracePeriod=30
Jun 03 23:29:10.185077 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.184954    2577 generic.go:356] "Generic (PLEG): container finished" podID="c9b06e42-99fa-47e2-847e-47b427f82c56" containerID="b6c87e819549784e3646dad0334d25efb1b981f1cee15bb1cb9110fc467a9b1a" exitCode=0
Jun 03 23:29:10.185198 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.185092    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9" event={"ID":"c9b06e42-99fa-47e2-847e-47b427f82c56","Type":"ContainerDied","Data":"b6c87e819549784e3646dad0334d25efb1b981f1cee15bb1cb9110fc467a9b1a"}
Jun 03 23:29:10.254905 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.254883    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:29:10.336782 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.336747    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6r5z\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-kube-api-access-x6r5z\") pod \"c9b06e42-99fa-47e2-847e-47b427f82c56\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") "
Jun 03 23:29:10.336782 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.336783    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls\") pod \"c9b06e42-99fa-47e2-847e-47b427f82c56\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") "
Jun 03 23:29:10.336998 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.336807    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-certificates\") pod \"c9b06e42-99fa-47e2-847e-47b427f82c56\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") "
Jun 03 23:29:10.336998 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.336847    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c9b06e42-99fa-47e2-847e-47b427f82c56-installation-pull-secrets\") pod \"c9b06e42-99fa-47e2-847e-47b427f82c56\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") "
Jun 03 23:29:10.336998 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.336868    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-bound-sa-token\") pod \"c9b06e42-99fa-47e2-847e-47b427f82c56\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") "
Jun 03 23:29:10.336998 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.336897    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c9b06e42-99fa-47e2-847e-47b427f82c56-image-registry-private-configuration\") pod \"c9b06e42-99fa-47e2-847e-47b427f82c56\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") "
Jun 03 23:29:10.336998 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.336946    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c9b06e42-99fa-47e2-847e-47b427f82c56-trusted-ca\") pod \"c9b06e42-99fa-47e2-847e-47b427f82c56\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") "
Jun 03 23:29:10.336998 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.336975    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c9b06e42-99fa-47e2-847e-47b427f82c56-ca-trust-extracted\") pod \"c9b06e42-99fa-47e2-847e-47b427f82c56\" (UID: \"c9b06e42-99fa-47e2-847e-47b427f82c56\") "
Jun 03 23:29:10.337823 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.337253    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c9b06e42-99fa-47e2-847e-47b427f82c56" (UID: "c9b06e42-99fa-47e2-847e-47b427f82c56"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:29:10.337823 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.337597    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9b06e42-99fa-47e2-847e-47b427f82c56-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c9b06e42-99fa-47e2-847e-47b427f82c56" (UID: "c9b06e42-99fa-47e2-847e-47b427f82c56"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:29:10.339394 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.339365    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9b06e42-99fa-47e2-847e-47b427f82c56-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c9b06e42-99fa-47e2-847e-47b427f82c56" (UID: "c9b06e42-99fa-47e2-847e-47b427f82c56"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 03 23:29:10.339517 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.339410    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c9b06e42-99fa-47e2-847e-47b427f82c56" (UID: "c9b06e42-99fa-47e2-847e-47b427f82c56"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 23:29:10.339517 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.339476    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c9b06e42-99fa-47e2-847e-47b427f82c56" (UID: "c9b06e42-99fa-47e2-847e-47b427f82c56"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 23:29:10.339629 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.339578    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-kube-api-access-x6r5z" (OuterVolumeSpecName: "kube-api-access-x6r5z") pod "c9b06e42-99fa-47e2-847e-47b427f82c56" (UID: "c9b06e42-99fa-47e2-847e-47b427f82c56"). InnerVolumeSpecName "kube-api-access-x6r5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 23:29:10.339795 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.339763    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9b06e42-99fa-47e2-847e-47b427f82c56-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c9b06e42-99fa-47e2-847e-47b427f82c56" (UID: "c9b06e42-99fa-47e2-847e-47b427f82c56"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 03 23:29:10.345455 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.345428    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9b06e42-99fa-47e2-847e-47b427f82c56-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c9b06e42-99fa-47e2-847e-47b427f82c56" (UID: "c9b06e42-99fa-47e2-847e-47b427f82c56"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 03 23:29:10.438467 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.438438    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-certificates\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:29:10.438467 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.438462    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c9b06e42-99fa-47e2-847e-47b427f82c56-installation-pull-secrets\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:29:10.438467 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.438473    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-bound-sa-token\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:29:10.438675 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.438484    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c9b06e42-99fa-47e2-847e-47b427f82c56-image-registry-private-configuration\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:29:10.438675 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.438503    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c9b06e42-99fa-47e2-847e-47b427f82c56-trusted-ca\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:29:10.438675 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.438511    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c9b06e42-99fa-47e2-847e-47b427f82c56-ca-trust-extracted\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:29:10.438675 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.438519    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6r5z\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-kube-api-access-x6r5z\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:29:10.438675 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:10.438527    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c9b06e42-99fa-47e2-847e-47b427f82c56-registry-tls\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:29:11.189878 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:11.189847    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9"
Jun 03 23:29:11.189878 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:11.189865    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78d9cb695b-4hdm9" event={"ID":"c9b06e42-99fa-47e2-847e-47b427f82c56","Type":"ContainerDied","Data":"9a7dca6b167a55777d2d6e80fc3c49ef00b66dddd55c96a907a9735c043b8277"}
Jun 03 23:29:11.190350 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:11.189919    2577 scope.go:117] "RemoveContainer" containerID="b6c87e819549784e3646dad0334d25efb1b981f1cee15bb1cb9110fc467a9b1a"
Jun 03 23:29:11.210943 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:11.210913    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78d9cb695b-4hdm9"]
Jun 03 23:29:11.214723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:11.214702    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-78d9cb695b-4hdm9"]
Jun 03 23:29:11.273085 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:11.273065    2577 scope.go:117] "RemoveContainer" containerID="52af4cfe51125380c67e17418fb859a216ef7bc1a16ff938fced93510c401896"
Jun 03 23:29:11.979679 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:11.979639    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-56b467f5ff-994kl"]
Jun 03 23:29:11.980011 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:11.979992    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9b06e42-99fa-47e2-847e-47b427f82c56" containerName="registry"
Jun 03 23:29:11.980083 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:11.980013    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9b06e42-99fa-47e2-847e-47b427f82c56" containerName="registry"
Jun 03 23:29:11.980129 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:11.980085    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9b06e42-99fa-47e2-847e-47b427f82c56" containerName="registry"
Jun 03 23:29:11.981812 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:11.981790    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-56b467f5ff-994kl"
Jun 03 23:29:11.984471 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:11.984447    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jun 03 23:29:11.984597 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:11.984580    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jun 03 23:29:11.984686 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:11.984584    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-dxzxq\""
Jun 03 23:29:11.995818 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:11.995792    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-56b467f5ff-994kl"]
Jun 03 23:29:12.050868 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:12.050827    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2fk6\" (UniqueName: \"kubernetes.io/projected/f15e8ee0-46c6-407f-a2c1-a531179f2a9e-kube-api-access-t2fk6\") pod \"downloads-56b467f5ff-994kl\" (UID: \"f15e8ee0-46c6-407f-a2c1-a531179f2a9e\") " pod="openshift-console/downloads-56b467f5ff-994kl"
Jun 03 23:29:12.152276 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:12.152240    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2fk6\" (UniqueName: \"kubernetes.io/projected/f15e8ee0-46c6-407f-a2c1-a531179f2a9e-kube-api-access-t2fk6\") pod \"downloads-56b467f5ff-994kl\" (UID: \"f15e8ee0-46c6-407f-a2c1-a531179f2a9e\") " pod="openshift-console/downloads-56b467f5ff-994kl"
Jun 03 23:29:12.161869 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:12.161838    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2fk6\" (UniqueName: \"kubernetes.io/projected/f15e8ee0-46c6-407f-a2c1-a531179f2a9e-kube-api-access-t2fk6\") pod \"downloads-56b467f5ff-994kl\" (UID: \"f15e8ee0-46c6-407f-a2c1-a531179f2a9e\") " pod="openshift-console/downloads-56b467f5ff-994kl"
Jun 03 23:29:12.195167 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:12.195138    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 03 23:29:12.195551 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:12.195235    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l" event={"ID":"397370f1-b682-48ca-b643-f447f300c2a5","Type":"ContainerStarted","Data":"640b2439fd50e68399317848348a64b0604b3c13dc970c8aca05ff7d63cbfd43"}
Jun 03 23:29:12.195551 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:12.195524    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:29:12.200385 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:12.200362    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l"
Jun 03 23:29:12.213604 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:12.213558    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85bd8fd97d-tsn6l" podStartSLOduration=55.99486513 podStartE2EDuration="58.213541281s" totalImagesPullingTime="2.218676151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:29:12.212821096 +0000 UTC m=+274.375499181" watchObservedRunningTime="2026-06-03 23:29:12.213541281 +0000 UTC m=+274.376219366"
Jun 03 23:29:12.278592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:12.278514    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9b06e42-99fa-47e2-847e-47b427f82c56" path="/var/lib/kubelet/pods/c9b06e42-99fa-47e2-847e-47b427f82c56/volumes"
Jun 03 23:29:12.292587 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:12.292390    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-56b467f5ff-994kl"
Jun 03 23:29:12.435274 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:12.435242    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-56b467f5ff-994kl"]
Jun 03 23:29:12.437134 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:29:12.437089    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf15e8ee0_46c6_407f_a2c1_a531179f2a9e.slice/crio-4c871fe0524ab05e4c0d800f7decc5b5699eb5bcba36cfd5da77ec3f322025c1 WatchSource:0}: Error finding container 4c871fe0524ab05e4c0d800f7decc5b5699eb5bcba36cfd5da77ec3f322025c1: Status 404 returned error can't find the container with id 4c871fe0524ab05e4c0d800f7decc5b5699eb5bcba36cfd5da77ec3f322025c1
Jun 03 23:29:13.201309 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:13.201269    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-56b467f5ff-994kl" event={"ID":"f15e8ee0-46c6-407f-a2c1-a531179f2a9e","Type":"ContainerStarted","Data":"4c871fe0524ab05e4c0d800f7decc5b5699eb5bcba36cfd5da77ec3f322025c1"}
Jun 03 23:29:17.814144 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:29:17.814091    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5q42g" podUID="3a1a6edf-938e-4784-bbc6-74c8829977a2"
Jun 03 23:29:18.221949 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:18.221869    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5q42g"
Jun 03 23:29:20.904372 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:20.904334    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f7d587db7-jtd8l"]
Jun 03 23:29:20.907538 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:20.907515    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:20.909937 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:20.909917    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jun 03 23:29:20.910678 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:20.910657    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jun 03 23:29:20.910678 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:20.910666    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jun 03 23:29:20.910832 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:20.910666    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jun 03 23:29:20.910832 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:20.910666    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bjct2\""
Jun 03 23:29:20.910832 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:20.910749    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jun 03 23:29:20.919039 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:20.919016    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f7d587db7-jtd8l"]
Jun 03 23:29:21.031348 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.031307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-oauth-serving-cert\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.031530 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.031364    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-config\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.031530 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.031398    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-oauth-config\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.031530 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.031455    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-serving-cert\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.031530 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.031502    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-service-ca\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.031723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.031534    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gf6r\" (UniqueName: \"kubernetes.io/projected/0d5e1ca0-14fe-4879-bd9c-16f465677ade-kube-api-access-8gf6r\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.132639 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.132597    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-oauth-serving-cert\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.132817 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.132652    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-config\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.132817 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.132678    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-oauth-config\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.132817 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.132718    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-serving-cert\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.132817 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.132756    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-service-ca\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.132817 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.132791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gf6r\" (UniqueName: \"kubernetes.io/projected/0d5e1ca0-14fe-4879-bd9c-16f465677ade-kube-api-access-8gf6r\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.133479 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.133448    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-config\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.133610 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.133532    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-oauth-serving-cert\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.133771 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.133730    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-service-ca\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.135428 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.135404    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-oauth-config\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.135548 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.135529    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-serving-cert\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.143622 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.143601    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gf6r\" (UniqueName: \"kubernetes.io/projected/0d5e1ca0-14fe-4879-bd9c-16f465677ade-kube-api-access-8gf6r\") pod \"console-f7d587db7-jtd8l\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") " pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.217727 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.217644    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:21.737726 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.737686    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:29:21.740323 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.740295    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a1a6edf-938e-4784-bbc6-74c8829977a2-metrics-tls\") pod \"dns-default-5q42g\" (UID: \"3a1a6edf-938e-4784-bbc6-74c8829977a2\") " pod="openshift-dns/dns-default-5q42g"
Jun 03 23:29:21.824504 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.824461    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6h75n\""
Jun 03 23:29:21.833155 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:21.833127    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5q42g"
Jun 03 23:29:26.625646 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:26.625581    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n99j6_e42566fd-c3a2-48b7-8d03-a172d908aeff/dns-node-resolver/0.log"
Jun 03 23:29:27.027064 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:27.026966    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55b9b48c7c-jw6cm_98372747-9a9e-4a1e-ad2f-d274d1db6889/router/0.log"
Jun 03 23:29:27.225790 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:27.225758    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-28f9z_201466a6-58e5-49a6-869f-2161d2c8b708/serve-healthcheck-canary/0.log"
Jun 03 23:29:27.978622 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:27.978596    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f7d587db7-jtd8l"]
Jun 03 23:29:27.989233 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:29:27.989183    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d5e1ca0_14fe_4879_bd9c_16f465677ade.slice/crio-362d9a512a51bd4dfe528bc587d0d799fafe2e1cc86dc2187057740c484e2fc0 WatchSource:0}: Error finding container 362d9a512a51bd4dfe528bc587d0d799fafe2e1cc86dc2187057740c484e2fc0: Status 404 returned error can't find the container with id 362d9a512a51bd4dfe528bc587d0d799fafe2e1cc86dc2187057740c484e2fc0
Jun 03 23:29:27.994953 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:27.994931    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5q42g"]
Jun 03 23:29:27.997559 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:29:27.997532    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a1a6edf_938e_4784_bbc6_74c8829977a2.slice/crio-1b410d6d79b4762178db7d3454f419a3df4181ad40904b3c2771cad0f0cc1367 WatchSource:0}: Error finding container 1b410d6d79b4762178db7d3454f419a3df4181ad40904b3c2771cad0f0cc1367: Status 404 returned error can't find the container with id 1b410d6d79b4762178db7d3454f419a3df4181ad40904b3c2771cad0f0cc1367
Jun 03 23:29:28.271410 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.271315    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f7d587db7-jtd8l" event={"ID":"0d5e1ca0-14fe-4879-bd9c-16f465677ade","Type":"ContainerStarted","Data":"362d9a512a51bd4dfe528bc587d0d799fafe2e1cc86dc2187057740c484e2fc0"}
Jun 03 23:29:28.278309 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.278278    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-56b467f5ff-994kl"
Jun 03 23:29:28.278462 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.278332    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5q42g" event={"ID":"3a1a6edf-938e-4784-bbc6-74c8829977a2","Type":"ContainerStarted","Data":"1b410d6d79b4762178db7d3454f419a3df4181ad40904b3c2771cad0f0cc1367"}
Jun 03 23:29:28.278462 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.278364    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-56b467f5ff-994kl" event={"ID":"f15e8ee0-46c6-407f-a2c1-a531179f2a9e","Type":"ContainerStarted","Data":"e5df17d737a17ca2fb7aa0317590b61cddda17581c628544d80bb00937da35c9"}
Jun 03 23:29:28.283034 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.283006    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74b446b88d-dpz5r"]
Jun 03 23:29:28.287005 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.286976    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-56b467f5ff-994kl"
Jun 03 23:29:28.287121 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.287064    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.293642 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.293619    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jun 03 23:29:28.297815 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.297763    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-56b467f5ff-994kl" podStartSLOduration=1.785710905 podStartE2EDuration="17.297746615s" totalImagesPullingTime="15.51203571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:29:28.296059957 +0000 UTC m=+290.458738032" watchObservedRunningTime="2026-06-03 23:29:28.297746615 +0000 UTC m=+290.460424690"
Jun 03 23:29:28.300122 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.300094    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74b446b88d-dpz5r"]
Jun 03 23:29:28.397192 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.397145    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55625080-cac3-4b3b-a202-5496edc698f5-console-oauth-config\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.397366 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.397218    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-trusted-ca-bundle\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.397366 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.397259    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55625080-cac3-4b3b-a202-5496edc698f5-console-serving-cert\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.397366 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.397310    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-oauth-serving-cert\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.397366 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.397353    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn9xq\" (UniqueName: \"kubernetes.io/projected/55625080-cac3-4b3b-a202-5496edc698f5-kube-api-access-sn9xq\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.397568 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.397392    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-console-config\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.397568 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.397418    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-service-ca\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.498795 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.498631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-console-config\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.498795 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.498680    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-service-ca\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.498795 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.498740    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55625080-cac3-4b3b-a202-5496edc698f5-console-oauth-config\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.498795 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.498769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-trusted-ca-bundle\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.498795 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.498797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55625080-cac3-4b3b-a202-5496edc698f5-console-serving-cert\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.499178 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.498845    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-oauth-serving-cert\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.499178 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.498883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn9xq\" (UniqueName: \"kubernetes.io/projected/55625080-cac3-4b3b-a202-5496edc698f5-kube-api-access-sn9xq\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.500022 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.499954    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-console-config\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.500722 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.500534    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-service-ca\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.501228 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.501082    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-oauth-serving-cert\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.501228 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.501085    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-trusted-ca-bundle\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.503498 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.503456    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55625080-cac3-4b3b-a202-5496edc698f5-console-oauth-config\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.503758 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.503737    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55625080-cac3-4b3b-a202-5496edc698f5-console-serving-cert\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.507619 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.507599    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn9xq\" (UniqueName: \"kubernetes.io/projected/55625080-cac3-4b3b-a202-5496edc698f5-kube-api-access-sn9xq\") pod \"console-74b446b88d-dpz5r\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") " pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.599187 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.598926    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:28.783452 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:28.783245    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74b446b88d-dpz5r"]
Jun 03 23:29:28.786711 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:29:28.786663    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55625080_cac3_4b3b_a202_5496edc698f5.slice/crio-8966bafe99679f76390648e04bddafdf8de89683fcee8cad9c2082d456ba554a WatchSource:0}: Error finding container 8966bafe99679f76390648e04bddafdf8de89683fcee8cad9c2082d456ba554a: Status 404 returned error can't find the container with id 8966bafe99679f76390648e04bddafdf8de89683fcee8cad9c2082d456ba554a
Jun 03 23:29:29.282860 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:29.282723    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74b446b88d-dpz5r" event={"ID":"55625080-cac3-4b3b-a202-5496edc698f5","Type":"ContainerStarted","Data":"8966bafe99679f76390648e04bddafdf8de89683fcee8cad9c2082d456ba554a"}
Jun 03 23:29:33.306347 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:33.306264    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f7d587db7-jtd8l" event={"ID":"0d5e1ca0-14fe-4879-bd9c-16f465677ade","Type":"ContainerStarted","Data":"e7a2e5b2fc78f85ab3276acd453cc323e890442a323f8a4bd18dd6739c423544"}
Jun 03 23:29:33.308127 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:33.308091    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74b446b88d-dpz5r" event={"ID":"55625080-cac3-4b3b-a202-5496edc698f5","Type":"ContainerStarted","Data":"09972639a121e7c6d61ee029d7259e4ef4d65df2f28c339ec4deb040e379595a"}
Jun 03 23:29:33.310355 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:33.310325    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5q42g" event={"ID":"3a1a6edf-938e-4784-bbc6-74c8829977a2","Type":"ContainerStarted","Data":"26c33528920c20379099cfca8fed72e99be175fc791e507f3d7ae64c018a5de0"}
Jun 03 23:29:33.310499 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:33.310377    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5q42g" event={"ID":"3a1a6edf-938e-4784-bbc6-74c8829977a2","Type":"ContainerStarted","Data":"c30b126abf78c388bea91347c9ad46ed6784f8e23956be108ad57d9cc438c36b"}
Jun 03 23:29:33.310499 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:33.310430    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5q42g"
Jun 03 23:29:33.325480 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:33.325425    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f7d587db7-jtd8l" podStartSLOduration=8.61943172 podStartE2EDuration="13.325410848s" totalImagesPullingTime="4.705979128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:29:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:29:33.32346823 +0000 UTC m=+295.486146326" watchObservedRunningTime="2026-06-03 23:29:33.325410848 +0000 UTC m=+295.488088925"
Jun 03 23:29:33.342241 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:33.342184    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5q42g" podStartSLOduration=258.113764771 podStartE2EDuration="4m22.342167577s" totalImagesPullingTime="4.228402806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:25:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:29:33.341342609 +0000 UTC m=+295.504020686" watchObservedRunningTime="2026-06-03 23:29:33.342167577 +0000 UTC m=+295.504845652"
Jun 03 23:29:33.363341 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:33.363299    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74b446b88d-dpz5r" podStartSLOduration=1.452595116 podStartE2EDuration="5.363284554s" totalImagesPullingTime="3.910689438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:29:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:29:33.36252397 +0000 UTC m=+295.525202047" watchObservedRunningTime="2026-06-03 23:29:33.363284554 +0000 UTC m=+295.525962631"
Jun 03 23:29:38.223929 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:38.223830    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 03 23:29:38.225602 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:38.225572    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 03 23:29:38.250091 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:38.250071    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jun 03 23:29:38.599190 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:38.599160    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:38.631914 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:38.599255    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:38.631914 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:38.604136    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:39.340838 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:39.340812    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:29:39.402867 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:39.402834    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f7d587db7-jtd8l"]
Jun 03 23:29:41.218191 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:41.218149    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:29:43.317906 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:29:43.317872    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5q42g"
Jun 03 23:30:04.423998 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.423943    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-f7d587db7-jtd8l" podUID="0d5e1ca0-14fe-4879-bd9c-16f465677ade" containerName="console" containerID="cri-o://e7a2e5b2fc78f85ab3276acd453cc323e890442a323f8a4bd18dd6739c423544" gracePeriod=15
Jun 03 23:30:04.692017 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.691953    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:30:04.809584 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.809551    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-serving-cert\") pod \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") "
Jun 03 23:30:04.809584 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.809588    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gf6r\" (UniqueName: \"kubernetes.io/projected/0d5e1ca0-14fe-4879-bd9c-16f465677ade-kube-api-access-8gf6r\") pod \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") "
Jun 03 23:30:04.809825 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.809620    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-oauth-config\") pod \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") "
Jun 03 23:30:04.809825 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.809649    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-config\") pod \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") "
Jun 03 23:30:04.809825 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.809674    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-service-ca\") pod \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") "
Jun 03 23:30:04.809825 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.809712    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-oauth-serving-cert\") pod \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\" (UID: \"0d5e1ca0-14fe-4879-bd9c-16f465677ade\") "
Jun 03 23:30:04.810100 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.810067    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-config" (OuterVolumeSpecName: "console-config") pod "0d5e1ca0-14fe-4879-bd9c-16f465677ade" (UID: "0d5e1ca0-14fe-4879-bd9c-16f465677ade"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:30:04.810190 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.810141    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-service-ca" (OuterVolumeSpecName: "service-ca") pod "0d5e1ca0-14fe-4879-bd9c-16f465677ade" (UID: "0d5e1ca0-14fe-4879-bd9c-16f465677ade"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:30:04.810269 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.810228    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0d5e1ca0-14fe-4879-bd9c-16f465677ade" (UID: "0d5e1ca0-14fe-4879-bd9c-16f465677ade"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:30:04.811921 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.811894    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0d5e1ca0-14fe-4879-bd9c-16f465677ade" (UID: "0d5e1ca0-14fe-4879-bd9c-16f465677ade"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 03 23:30:04.812261 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.812192    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0d5e1ca0-14fe-4879-bd9c-16f465677ade" (UID: "0d5e1ca0-14fe-4879-bd9c-16f465677ade"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 03 23:30:04.812261 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.812194    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d5e1ca0-14fe-4879-bd9c-16f465677ade-kube-api-access-8gf6r" (OuterVolumeSpecName: "kube-api-access-8gf6r") pod "0d5e1ca0-14fe-4879-bd9c-16f465677ade" (UID: "0d5e1ca0-14fe-4879-bd9c-16f465677ade"). InnerVolumeSpecName "kube-api-access-8gf6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 23:30:04.910759 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.910716    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-serving-cert\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:30:04.910759 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.910752    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gf6r\" (UniqueName: \"kubernetes.io/projected/0d5e1ca0-14fe-4879-bd9c-16f465677ade-kube-api-access-8gf6r\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:30:04.910759 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.910762    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-oauth-config\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:30:04.910759 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.910771    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-console-config\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:30:04.911032 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.910779    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-service-ca\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:30:04.911032 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:04.910788    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d5e1ca0-14fe-4879-bd9c-16f465677ade-oauth-serving-cert\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:30:05.434885 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:05.434851    2577 generic.go:356] "Generic (PLEG): container finished" podID="0d5e1ca0-14fe-4879-bd9c-16f465677ade" containerID="e7a2e5b2fc78f85ab3276acd453cc323e890442a323f8a4bd18dd6739c423544" exitCode=0
Jun 03 23:30:05.435288 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:05.434895    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f7d587db7-jtd8l"
Jun 03 23:30:05.435288 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:05.434917    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f7d587db7-jtd8l" event={"ID":"0d5e1ca0-14fe-4879-bd9c-16f465677ade","Type":"ContainerDied","Data":"e7a2e5b2fc78f85ab3276acd453cc323e890442a323f8a4bd18dd6739c423544"}
Jun 03 23:30:05.435288 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:05.434955    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f7d587db7-jtd8l" event={"ID":"0d5e1ca0-14fe-4879-bd9c-16f465677ade","Type":"ContainerDied","Data":"362d9a512a51bd4dfe528bc587d0d799fafe2e1cc86dc2187057740c484e2fc0"}
Jun 03 23:30:05.435288 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:05.434974    2577 scope.go:117] "RemoveContainer" containerID="e7a2e5b2fc78f85ab3276acd453cc323e890442a323f8a4bd18dd6739c423544"
Jun 03 23:30:05.447831 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:05.447812    2577 scope.go:117] "RemoveContainer" containerID="e7a2e5b2fc78f85ab3276acd453cc323e890442a323f8a4bd18dd6739c423544"
Jun 03 23:30:05.448093 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:30:05.448073    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7a2e5b2fc78f85ab3276acd453cc323e890442a323f8a4bd18dd6739c423544\": container with ID starting with e7a2e5b2fc78f85ab3276acd453cc323e890442a323f8a4bd18dd6739c423544 not found: ID does not exist" containerID="e7a2e5b2fc78f85ab3276acd453cc323e890442a323f8a4bd18dd6739c423544"
Jun 03 23:30:05.448157 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:05.448104    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7a2e5b2fc78f85ab3276acd453cc323e890442a323f8a4bd18dd6739c423544"} err="failed to get container status \"e7a2e5b2fc78f85ab3276acd453cc323e890442a323f8a4bd18dd6739c423544\": rpc error: code = NotFound desc = could not find container \"e7a2e5b2fc78f85ab3276acd453cc323e890442a323f8a4bd18dd6739c423544\": container with ID starting with e7a2e5b2fc78f85ab3276acd453cc323e890442a323f8a4bd18dd6739c423544 not found: ID does not exist"
Jun 03 23:30:05.457913 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:05.457889    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f7d587db7-jtd8l"]
Jun 03 23:30:05.461982 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:05.461962    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f7d587db7-jtd8l"]
Jun 03 23:30:06.277762 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:06.277730    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d5e1ca0-14fe-4879-bd9c-16f465677ade" path="/var/lib/kubelet/pods/0d5e1ca0-14fe-4879-bd9c-16f465677ade/volumes"
Jun 03 23:30:26.667606 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.667517    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b44d45f65-pxnth"]
Jun 03 23:30:26.668507 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.668477    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d5e1ca0-14fe-4879-bd9c-16f465677ade" containerName="console"
Jun 03 23:30:26.668507 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.668507    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d5e1ca0-14fe-4879-bd9c-16f465677ade" containerName="console"
Jun 03 23:30:26.668671 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.668585    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d5e1ca0-14fe-4879-bd9c-16f465677ade" containerName="console"
Jun 03 23:30:26.670701 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.670675    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.686857 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.686832    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b44d45f65-pxnth"]
Jun 03 23:30:26.784101 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.784058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-console-config\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.784297 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.784126    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c162c939-405b-49ec-b9b1-260346c730c6-console-oauth-config\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.784297 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.784158    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-trusted-ca-bundle\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.784297 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.784232    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-service-ca\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.784297 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.784256    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-oauth-serving-cert\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.784518 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.784288    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c162c939-405b-49ec-b9b1-260346c730c6-console-serving-cert\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.784518 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.784350    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bgkm\" (UniqueName: \"kubernetes.io/projected/c162c939-405b-49ec-b9b1-260346c730c6-kube-api-access-4bgkm\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.885396 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.885360    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bgkm\" (UniqueName: \"kubernetes.io/projected/c162c939-405b-49ec-b9b1-260346c730c6-kube-api-access-4bgkm\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.885396 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.885414    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-console-config\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.885596 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.885531    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c162c939-405b-49ec-b9b1-260346c730c6-console-oauth-config\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.885596 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.885565    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-trusted-ca-bundle\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.885697 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.885601    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-service-ca\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.885697 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.885625    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-oauth-serving-cert\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.885697 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.885662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c162c939-405b-49ec-b9b1-260346c730c6-console-serving-cert\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.886193 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.886165    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-console-config\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.886381 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.886357    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-service-ca\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.886435 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.886388    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-oauth-serving-cert\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.886779 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.886755    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-trusted-ca-bundle\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.888043 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.888018    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c162c939-405b-49ec-b9b1-260346c730c6-console-serving-cert\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.888130 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.888081    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c162c939-405b-49ec-b9b1-260346c730c6-console-oauth-config\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.893007 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.892986    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bgkm\" (UniqueName: \"kubernetes.io/projected/c162c939-405b-49ec-b9b1-260346c730c6-kube-api-access-4bgkm\") pod \"console-6b44d45f65-pxnth\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") " pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:26.983089 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:26.982766    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:27.122803 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:27.122775    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b44d45f65-pxnth"]
Jun 03 23:30:27.125267 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:30:27.125235    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc162c939_405b_49ec_b9b1_260346c730c6.slice/crio-38e637dc5b2e1051eefb4a8222f7f6acd2017caf7db0d0bc0a0e95031eeb3df1 WatchSource:0}: Error finding container 38e637dc5b2e1051eefb4a8222f7f6acd2017caf7db0d0bc0a0e95031eeb3df1: Status 404 returned error can't find the container with id 38e637dc5b2e1051eefb4a8222f7f6acd2017caf7db0d0bc0a0e95031eeb3df1
Jun 03 23:30:27.516981 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:27.516938    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b44d45f65-pxnth" event={"ID":"c162c939-405b-49ec-b9b1-260346c730c6","Type":"ContainerStarted","Data":"0bd7e7256c40a25758c002f283e7a0272f050667d6722a250594e5ac90a9fb55"}
Jun 03 23:30:27.517129 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:27.516993    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b44d45f65-pxnth" event={"ID":"c162c939-405b-49ec-b9b1-260346c730c6","Type":"ContainerStarted","Data":"38e637dc5b2e1051eefb4a8222f7f6acd2017caf7db0d0bc0a0e95031eeb3df1"}
Jun 03 23:30:27.535632 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:27.535582    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b44d45f65-pxnth" podStartSLOduration=1.5355682449999999 podStartE2EDuration="1.535568245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:30:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:30:27.534250741 +0000 UTC m=+349.696928815" watchObservedRunningTime="2026-06-03 23:30:27.535568245 +0000 UTC m=+349.698246319"
Jun 03 23:30:36.984234 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:36.984181    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:36.984643 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:36.984276    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:36.988955 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:36.988931    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:37.556952 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:37.556928    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:30:37.603015 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:30:37.602985    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74b446b88d-dpz5r"]
Jun 03 23:31:02.629245 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:02.629172    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-74b446b88d-dpz5r" podUID="55625080-cac3-4b3b-a202-5496edc698f5" containerName="console" containerID="cri-o://09972639a121e7c6d61ee029d7259e4ef4d65df2f28c339ec4deb040e379595a" gracePeriod=15
Jun 03 23:31:02.865662 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:02.865636    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:31:02.961421 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:02.961336    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55625080-cac3-4b3b-a202-5496edc698f5-console-oauth-config\") pod \"55625080-cac3-4b3b-a202-5496edc698f5\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") "
Jun 03 23:31:02.961421 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:02.961375    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-service-ca\") pod \"55625080-cac3-4b3b-a202-5496edc698f5\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") "
Jun 03 23:31:02.961421 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:02.961398    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-trusted-ca-bundle\") pod \"55625080-cac3-4b3b-a202-5496edc698f5\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") "
Jun 03 23:31:02.961687 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:02.961438    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-console-config\") pod \"55625080-cac3-4b3b-a202-5496edc698f5\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") "
Jun 03 23:31:02.961687 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:02.961465    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sn9xq\" (UniqueName: \"kubernetes.io/projected/55625080-cac3-4b3b-a202-5496edc698f5-kube-api-access-sn9xq\") pod \"55625080-cac3-4b3b-a202-5496edc698f5\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") "
Jun 03 23:31:02.961687 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:02.961486    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55625080-cac3-4b3b-a202-5496edc698f5-console-serving-cert\") pod \"55625080-cac3-4b3b-a202-5496edc698f5\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") "
Jun 03 23:31:02.961687 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:02.961502    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-oauth-serving-cert\") pod \"55625080-cac3-4b3b-a202-5496edc698f5\" (UID: \"55625080-cac3-4b3b-a202-5496edc698f5\") "
Jun 03 23:31:02.961953 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:02.961814    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-service-ca" (OuterVolumeSpecName: "service-ca") pod "55625080-cac3-4b3b-a202-5496edc698f5" (UID: "55625080-cac3-4b3b-a202-5496edc698f5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:31:02.962067 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:02.961952    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-console-config" (OuterVolumeSpecName: "console-config") pod "55625080-cac3-4b3b-a202-5496edc698f5" (UID: "55625080-cac3-4b3b-a202-5496edc698f5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:31:02.962067 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:02.962004    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "55625080-cac3-4b3b-a202-5496edc698f5" (UID: "55625080-cac3-4b3b-a202-5496edc698f5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:31:02.962067 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:02.962046    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "55625080-cac3-4b3b-a202-5496edc698f5" (UID: "55625080-cac3-4b3b-a202-5496edc698f5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:31:02.963698 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:02.963670    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55625080-cac3-4b3b-a202-5496edc698f5-kube-api-access-sn9xq" (OuterVolumeSpecName: "kube-api-access-sn9xq") pod "55625080-cac3-4b3b-a202-5496edc698f5" (UID: "55625080-cac3-4b3b-a202-5496edc698f5"). InnerVolumeSpecName "kube-api-access-sn9xq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 23:31:02.964051 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:02.964024    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55625080-cac3-4b3b-a202-5496edc698f5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "55625080-cac3-4b3b-a202-5496edc698f5" (UID: "55625080-cac3-4b3b-a202-5496edc698f5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 03 23:31:02.964149 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:02.964033    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55625080-cac3-4b3b-a202-5496edc698f5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "55625080-cac3-4b3b-a202-5496edc698f5" (UID: "55625080-cac3-4b3b-a202-5496edc698f5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 03 23:31:03.062313 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:03.062271    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-console-config\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:31:03.062313 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:03.062305    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn9xq\" (UniqueName: \"kubernetes.io/projected/55625080-cac3-4b3b-a202-5496edc698f5-kube-api-access-sn9xq\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:31:03.062313 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:03.062318    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55625080-cac3-4b3b-a202-5496edc698f5-console-serving-cert\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:31:03.062557 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:03.062331    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-oauth-serving-cert\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:31:03.062557 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:03.062342    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55625080-cac3-4b3b-a202-5496edc698f5-console-oauth-config\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:31:03.062557 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:03.062353    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-service-ca\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:31:03.062557 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:03.062364    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55625080-cac3-4b3b-a202-5496edc698f5-trusted-ca-bundle\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:31:03.646959 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:03.646919    2577 generic.go:356] "Generic (PLEG): container finished" podID="55625080-cac3-4b3b-a202-5496edc698f5" containerID="09972639a121e7c6d61ee029d7259e4ef4d65df2f28c339ec4deb040e379595a" exitCode=0
Jun 03 23:31:03.647367 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:03.646988    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74b446b88d-dpz5r"
Jun 03 23:31:03.647367 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:03.646991    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74b446b88d-dpz5r" event={"ID":"55625080-cac3-4b3b-a202-5496edc698f5","Type":"ContainerDied","Data":"09972639a121e7c6d61ee029d7259e4ef4d65df2f28c339ec4deb040e379595a"}
Jun 03 23:31:03.647367 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:03.647043    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74b446b88d-dpz5r" event={"ID":"55625080-cac3-4b3b-a202-5496edc698f5","Type":"ContainerDied","Data":"8966bafe99679f76390648e04bddafdf8de89683fcee8cad9c2082d456ba554a"}
Jun 03 23:31:03.647367 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:03.647072    2577 scope.go:117] "RemoveContainer" containerID="09972639a121e7c6d61ee029d7259e4ef4d65df2f28c339ec4deb040e379595a"
Jun 03 23:31:03.655360 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:03.655340    2577 scope.go:117] "RemoveContainer" containerID="09972639a121e7c6d61ee029d7259e4ef4d65df2f28c339ec4deb040e379595a"
Jun 03 23:31:03.655602 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:31:03.655584    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09972639a121e7c6d61ee029d7259e4ef4d65df2f28c339ec4deb040e379595a\": container with ID starting with 09972639a121e7c6d61ee029d7259e4ef4d65df2f28c339ec4deb040e379595a not found: ID does not exist" containerID="09972639a121e7c6d61ee029d7259e4ef4d65df2f28c339ec4deb040e379595a"
Jun 03 23:31:03.655640 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:03.655610    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09972639a121e7c6d61ee029d7259e4ef4d65df2f28c339ec4deb040e379595a"} err="failed to get container status \"09972639a121e7c6d61ee029d7259e4ef4d65df2f28c339ec4deb040e379595a\": rpc error: code = NotFound desc = could not find container \"09972639a121e7c6d61ee029d7259e4ef4d65df2f28c339ec4deb040e379595a\": container with ID starting with 09972639a121e7c6d61ee029d7259e4ef4d65df2f28c339ec4deb040e379595a not found: ID does not exist"
Jun 03 23:31:03.669424 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:03.669399    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74b446b88d-dpz5r"]
Jun 03 23:31:03.673156 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:03.673137    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74b446b88d-dpz5r"]
Jun 03 23:31:04.277965 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:04.277923    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55625080-cac3-4b3b-a202-5496edc698f5" path="/var/lib/kubelet/pods/55625080-cac3-4b3b-a202-5496edc698f5/volumes"
Jun 03 23:31:47.486260 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.486225    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56bcd9f976-nvqrf"]
Jun 03 23:31:47.486716 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.486487    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55625080-cac3-4b3b-a202-5496edc698f5" containerName="console"
Jun 03 23:31:47.486716 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.486498    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="55625080-cac3-4b3b-a202-5496edc698f5" containerName="console"
Jun 03 23:31:47.486716 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.486563    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="55625080-cac3-4b3b-a202-5496edc698f5" containerName="console"
Jun 03 23:31:47.489179 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.489152    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.500588 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.500563    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56bcd9f976-nvqrf"]
Jun 03 23:31:47.602033 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.601996    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-config\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.602033 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.602034    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-oauth-config\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.602294 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.602052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-oauth-serving-cert\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.602294 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.602112    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-service-ca\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.602294 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.602228    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt4fb\" (UniqueName: \"kubernetes.io/projected/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-kube-api-access-bt4fb\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.602294 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.602266    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-trusted-ca-bundle\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.602294 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.602283    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-serving-cert\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.703559 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.703525    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-service-ca\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.703731 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.703570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt4fb\" (UniqueName: \"kubernetes.io/projected/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-kube-api-access-bt4fb\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.703731 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.703591    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-trusted-ca-bundle\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.703731 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.703606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-serving-cert\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.703731 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.703642    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-config\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.703731 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.703663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-oauth-config\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.703731 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.703679    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-oauth-serving-cert\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.704354 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.704328    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-service-ca\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.704492 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.704469    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-config\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.704547 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.704469    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-oauth-serving-cert\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.704584 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.704549    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-trusted-ca-bundle\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.706105 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.706086    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-oauth-config\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.706253 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.706234    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-serving-cert\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.711452 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.711431    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt4fb\" (UniqueName: \"kubernetes.io/projected/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-kube-api-access-bt4fb\") pod \"console-56bcd9f976-nvqrf\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") " pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.798894 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.798818    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:47.921158 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:47.921129    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56bcd9f976-nvqrf"]
Jun 03 23:31:47.923840 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:31:47.923815    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c058f12_b8e2_42c6_9ef0_0b1b98aa6ad2.slice/crio-af36527414e283e189071c09d0591b710bef06adc87cd55257e6e7d04bd1ca83 WatchSource:0}: Error finding container af36527414e283e189071c09d0591b710bef06adc87cd55257e6e7d04bd1ca83: Status 404 returned error can't find the container with id af36527414e283e189071c09d0591b710bef06adc87cd55257e6e7d04bd1ca83
Jun 03 23:31:48.808856 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:48.808819    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56bcd9f976-nvqrf" event={"ID":"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2","Type":"ContainerStarted","Data":"82b2b653eb0fcf94404f7e044547bc6cb6c961719040ad6ddb1a8d836c9b949a"}
Jun 03 23:31:48.809276 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:48.808871    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56bcd9f976-nvqrf" event={"ID":"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2","Type":"ContainerStarted","Data":"af36527414e283e189071c09d0591b710bef06adc87cd55257e6e7d04bd1ca83"}
Jun 03 23:31:48.828245 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:48.828178    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56bcd9f976-nvqrf" podStartSLOduration=1.8281619359999999 podStartE2EDuration="1.828161936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:31:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:31:48.827840081 +0000 UTC m=+430.990518157" watchObservedRunningTime="2026-06-03 23:31:48.828161936 +0000 UTC m=+430.990840010"
Jun 03 23:31:57.799367 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:57.799283    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:57.799367 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:57.799338    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:57.805431 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:57.805405    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:57.844752 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:57.844726    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:31:57.893041 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:31:57.893008    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b44d45f65-pxnth"]
Jun 03 23:32:22.915927 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:22.915883    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6b44d45f65-pxnth" podUID="c162c939-405b-49ec-b9b1-260346c730c6" containerName="console" containerID="cri-o://0bd7e7256c40a25758c002f283e7a0272f050667d6722a250594e5ac90a9fb55" gracePeriod=15
Jun 03 23:32:23.152710 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.152685    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:32:23.164855 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.164833    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bgkm\" (UniqueName: \"kubernetes.io/projected/c162c939-405b-49ec-b9b1-260346c730c6-kube-api-access-4bgkm\") pod \"c162c939-405b-49ec-b9b1-260346c730c6\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") "
Jun 03 23:32:23.164942 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.164875    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c162c939-405b-49ec-b9b1-260346c730c6-console-serving-cert\") pod \"c162c939-405b-49ec-b9b1-260346c730c6\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") "
Jun 03 23:32:23.164942 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.164896    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-oauth-serving-cert\") pod \"c162c939-405b-49ec-b9b1-260346c730c6\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") "
Jun 03 23:32:23.164942 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.164917    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c162c939-405b-49ec-b9b1-260346c730c6-console-oauth-config\") pod \"c162c939-405b-49ec-b9b1-260346c730c6\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") "
Jun 03 23:32:23.165094 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.164949    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-console-config\") pod \"c162c939-405b-49ec-b9b1-260346c730c6\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") "
Jun 03 23:32:23.165094 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.164976    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-trusted-ca-bundle\") pod \"c162c939-405b-49ec-b9b1-260346c730c6\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") "
Jun 03 23:32:23.165094 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.164995    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-service-ca\") pod \"c162c939-405b-49ec-b9b1-260346c730c6\" (UID: \"c162c939-405b-49ec-b9b1-260346c730c6\") "
Jun 03 23:32:23.165399 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.165368    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c162c939-405b-49ec-b9b1-260346c730c6" (UID: "c162c939-405b-49ec-b9b1-260346c730c6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:32:23.165491 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.165396    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-console-config" (OuterVolumeSpecName: "console-config") pod "c162c939-405b-49ec-b9b1-260346c730c6" (UID: "c162c939-405b-49ec-b9b1-260346c730c6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:32:23.165491 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.165423    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c162c939-405b-49ec-b9b1-260346c730c6" (UID: "c162c939-405b-49ec-b9b1-260346c730c6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:32:23.165491 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.165453    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-service-ca" (OuterVolumeSpecName: "service-ca") pod "c162c939-405b-49ec-b9b1-260346c730c6" (UID: "c162c939-405b-49ec-b9b1-260346c730c6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:32:23.166990 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.166931    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c162c939-405b-49ec-b9b1-260346c730c6-kube-api-access-4bgkm" (OuterVolumeSpecName: "kube-api-access-4bgkm") pod "c162c939-405b-49ec-b9b1-260346c730c6" (UID: "c162c939-405b-49ec-b9b1-260346c730c6"). InnerVolumeSpecName "kube-api-access-4bgkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 23:32:23.167073 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.167031    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c162c939-405b-49ec-b9b1-260346c730c6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c162c939-405b-49ec-b9b1-260346c730c6" (UID: "c162c939-405b-49ec-b9b1-260346c730c6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 03 23:32:23.167073 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.167042    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c162c939-405b-49ec-b9b1-260346c730c6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c162c939-405b-49ec-b9b1-260346c730c6" (UID: "c162c939-405b-49ec-b9b1-260346c730c6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 03 23:32:23.265930 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.265889    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-oauth-serving-cert\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:32:23.265930 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.265923    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c162c939-405b-49ec-b9b1-260346c730c6-console-oauth-config\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:32:23.265930 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.265933    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-console-config\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:32:23.265930 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.265942    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-trusted-ca-bundle\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:32:23.266188 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.265951    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c162c939-405b-49ec-b9b1-260346c730c6-service-ca\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:32:23.266188 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.265959    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bgkm\" (UniqueName: \"kubernetes.io/projected/c162c939-405b-49ec-b9b1-260346c730c6-kube-api-access-4bgkm\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:32:23.266188 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.265968    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c162c939-405b-49ec-b9b1-260346c730c6-console-serving-cert\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:32:23.932459 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.932418    2577 generic.go:356] "Generic (PLEG): container finished" podID="c162c939-405b-49ec-b9b1-260346c730c6" containerID="0bd7e7256c40a25758c002f283e7a0272f050667d6722a250594e5ac90a9fb55" exitCode=0
Jun 03 23:32:23.932459 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.932463    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b44d45f65-pxnth"
Jun 03 23:32:23.932902 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.932497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b44d45f65-pxnth" event={"ID":"c162c939-405b-49ec-b9b1-260346c730c6","Type":"ContainerDied","Data":"0bd7e7256c40a25758c002f283e7a0272f050667d6722a250594e5ac90a9fb55"}
Jun 03 23:32:23.932902 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.932548    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b44d45f65-pxnth" event={"ID":"c162c939-405b-49ec-b9b1-260346c730c6","Type":"ContainerDied","Data":"38e637dc5b2e1051eefb4a8222f7f6acd2017caf7db0d0bc0a0e95031eeb3df1"}
Jun 03 23:32:23.932902 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.932568    2577 scope.go:117] "RemoveContainer" containerID="0bd7e7256c40a25758c002f283e7a0272f050667d6722a250594e5ac90a9fb55"
Jun 03 23:32:23.941060 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.941041    2577 scope.go:117] "RemoveContainer" containerID="0bd7e7256c40a25758c002f283e7a0272f050667d6722a250594e5ac90a9fb55"
Jun 03 23:32:23.941351 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:32:23.941331    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bd7e7256c40a25758c002f283e7a0272f050667d6722a250594e5ac90a9fb55\": container with ID starting with 0bd7e7256c40a25758c002f283e7a0272f050667d6722a250594e5ac90a9fb55 not found: ID does not exist" containerID="0bd7e7256c40a25758c002f283e7a0272f050667d6722a250594e5ac90a9fb55"
Jun 03 23:32:23.941429 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.941358    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bd7e7256c40a25758c002f283e7a0272f050667d6722a250594e5ac90a9fb55"} err="failed to get container status \"0bd7e7256c40a25758c002f283e7a0272f050667d6722a250594e5ac90a9fb55\": rpc error: code = NotFound desc = could not find container \"0bd7e7256c40a25758c002f283e7a0272f050667d6722a250594e5ac90a9fb55\": container with ID starting with 0bd7e7256c40a25758c002f283e7a0272f050667d6722a250594e5ac90a9fb55 not found: ID does not exist"
Jun 03 23:32:23.954295 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.954265    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b44d45f65-pxnth"]
Jun 03 23:32:23.959768 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:23.959746    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b44d45f65-pxnth"]
Jun 03 23:32:24.278291 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:32:24.278195    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c162c939-405b-49ec-b9b1-260346c730c6" path="/var/lib/kubelet/pods/c162c939-405b-49ec-b9b1-260346c730c6/volumes"
Jun 03 23:33:00.489440 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.489401    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-db486"]
Jun 03 23:33:00.490027 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.489786    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c162c939-405b-49ec-b9b1-260346c730c6" containerName="console"
Jun 03 23:33:00.490027 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.489803    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c162c939-405b-49ec-b9b1-260346c730c6" containerName="console"
Jun 03 23:33:00.490027 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.489867    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c162c939-405b-49ec-b9b1-260346c730c6" containerName="console"
Jun 03 23:33:00.491662 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.491640    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-db486"
Jun 03 23:33:00.495839 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.495817    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jun 03 23:33:00.496482 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.496458    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-ztgks\""
Jun 03 23:33:00.496482 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.496475    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jun 03 23:33:00.506538 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.506513    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-db486"]
Jun 03 23:33:00.545252 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.545222    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba155b08-7f8d-4328-89cc-e383f8147c3c-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-db486\" (UID: \"ba155b08-7f8d-4328-89cc-e383f8147c3c\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-db486"
Jun 03 23:33:00.545396 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.545283    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq4hr\" (UniqueName: \"kubernetes.io/projected/ba155b08-7f8d-4328-89cc-e383f8147c3c-kube-api-access-xq4hr\") pod \"cert-manager-webhook-7dc76cbb94-db486\" (UID: \"ba155b08-7f8d-4328-89cc-e383f8147c3c\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-db486"
Jun 03 23:33:00.646637 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.646605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq4hr\" (UniqueName: \"kubernetes.io/projected/ba155b08-7f8d-4328-89cc-e383f8147c3c-kube-api-access-xq4hr\") pod \"cert-manager-webhook-7dc76cbb94-db486\" (UID: \"ba155b08-7f8d-4328-89cc-e383f8147c3c\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-db486"
Jun 03 23:33:00.646781 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.646671    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba155b08-7f8d-4328-89cc-e383f8147c3c-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-db486\" (UID: \"ba155b08-7f8d-4328-89cc-e383f8147c3c\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-db486"
Jun 03 23:33:00.655053 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.655026    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba155b08-7f8d-4328-89cc-e383f8147c3c-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-db486\" (UID: \"ba155b08-7f8d-4328-89cc-e383f8147c3c\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-db486"
Jun 03 23:33:00.655744 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.655724    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq4hr\" (UniqueName: \"kubernetes.io/projected/ba155b08-7f8d-4328-89cc-e383f8147c3c-kube-api-access-xq4hr\") pod \"cert-manager-webhook-7dc76cbb94-db486\" (UID: \"ba155b08-7f8d-4328-89cc-e383f8147c3c\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-db486"
Jun 03 23:33:00.799960 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.799873    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-db486"
Jun 03 23:33:00.923681 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.923651    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-db486"]
Jun 03 23:33:00.928036 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:33:00.928009    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba155b08_7f8d_4328_89cc_e383f8147c3c.slice/crio-c2be59f31025e3854d0c673b059f4da2c0142b639567e801605c4e212aa78ae9 WatchSource:0}: Error finding container c2be59f31025e3854d0c673b059f4da2c0142b639567e801605c4e212aa78ae9: Status 404 returned error can't find the container with id c2be59f31025e3854d0c673b059f4da2c0142b639567e801605c4e212aa78ae9
Jun 03 23:33:00.929893 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:00.929873    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 03 23:33:01.062767 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:01.062676    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-db486" event={"ID":"ba155b08-7f8d-4328-89cc-e383f8147c3c","Type":"ContainerStarted","Data":"c2be59f31025e3854d0c673b059f4da2c0142b639567e801605c4e212aa78ae9"}
Jun 03 23:33:04.075170 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:04.075135    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-db486" event={"ID":"ba155b08-7f8d-4328-89cc-e383f8147c3c","Type":"ContainerStarted","Data":"87da650b89da2ed2544f4f668111416fa33776bf0837895abb850283d7fa2776"}
Jun 03 23:33:04.075666 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:04.075262    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-db486"
Jun 03 23:33:04.091502 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:04.091454    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-db486" podStartSLOduration=1.5062177810000001 podStartE2EDuration="4.091440454s" totalImagesPullingTime="2.585222673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:33:04.090982631 +0000 UTC m=+506.253660710" watchObservedRunningTime="2026-06-03 23:33:04.091440454 +0000 UTC m=+506.254118531"
Jun 03 23:33:07.012712 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:07.012673    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-7tnvh"]
Jun 03 23:33:07.014971 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:07.014953    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-7tnvh"
Jun 03 23:33:07.019072 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:07.019055    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-p6pxp\""
Jun 03 23:33:07.027720 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:07.027692    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-7tnvh"]
Jun 03 23:33:07.103576 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:07.103540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed5db117-5064-4e12-8e23-61357392945d-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-7tnvh\" (UID: \"ed5db117-5064-4e12-8e23-61357392945d\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-7tnvh"
Jun 03 23:33:07.103576 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:07.103573    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpgs2\" (UniqueName: \"kubernetes.io/projected/ed5db117-5064-4e12-8e23-61357392945d-kube-api-access-tpgs2\") pod \"cert-manager-cainjector-5d99449ffd-7tnvh\" (UID: \"ed5db117-5064-4e12-8e23-61357392945d\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-7tnvh"
Jun 03 23:33:07.204622 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:07.204583    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed5db117-5064-4e12-8e23-61357392945d-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-7tnvh\" (UID: \"ed5db117-5064-4e12-8e23-61357392945d\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-7tnvh"
Jun 03 23:33:07.204622 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:07.204621    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpgs2\" (UniqueName: \"kubernetes.io/projected/ed5db117-5064-4e12-8e23-61357392945d-kube-api-access-tpgs2\") pod \"cert-manager-cainjector-5d99449ffd-7tnvh\" (UID: \"ed5db117-5064-4e12-8e23-61357392945d\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-7tnvh"
Jun 03 23:33:07.213654 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:07.213619    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed5db117-5064-4e12-8e23-61357392945d-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-7tnvh\" (UID: \"ed5db117-5064-4e12-8e23-61357392945d\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-7tnvh"
Jun 03 23:33:07.213912 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:07.213889    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpgs2\" (UniqueName: \"kubernetes.io/projected/ed5db117-5064-4e12-8e23-61357392945d-kube-api-access-tpgs2\") pod \"cert-manager-cainjector-5d99449ffd-7tnvh\" (UID: \"ed5db117-5064-4e12-8e23-61357392945d\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-7tnvh"
Jun 03 23:33:07.324388 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:07.324353    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-7tnvh"
Jun 03 23:33:07.448849 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:07.448816    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-7tnvh"]
Jun 03 23:33:07.451963 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:33:07.451934    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded5db117_5064_4e12_8e23_61357392945d.slice/crio-4dca65ec77796c9fd349d1350a3af7aacd2cf2f9db4e41268f34cea7c112db42 WatchSource:0}: Error finding container 4dca65ec77796c9fd349d1350a3af7aacd2cf2f9db4e41268f34cea7c112db42: Status 404 returned error can't find the container with id 4dca65ec77796c9fd349d1350a3af7aacd2cf2f9db4e41268f34cea7c112db42
Jun 03 23:33:08.090720 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:08.090688    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-7tnvh" event={"ID":"ed5db117-5064-4e12-8e23-61357392945d","Type":"ContainerStarted","Data":"adff5e91429acc574256d9cd4db73ad8c927d5649363c274ac89b5ac4d9995bb"}
Jun 03 23:33:08.091087 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:08.090729    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-7tnvh" event={"ID":"ed5db117-5064-4e12-8e23-61357392945d","Type":"ContainerStarted","Data":"4dca65ec77796c9fd349d1350a3af7aacd2cf2f9db4e41268f34cea7c112db42"}
Jun 03 23:33:08.106369 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:08.106320    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5d99449ffd-7tnvh" podStartSLOduration=2.106301636 podStartE2EDuration="2.106301636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:33:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:33:08.106062294 +0000 UTC m=+510.268740370" watchObservedRunningTime="2026-06-03 23:33:08.106301636 +0000 UTC m=+510.268979710"
Jun 03 23:33:10.080731 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:10.080700    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-db486"
Jun 03 23:33:20.094568 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:20.094533    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-59c77684cc-pgqk6"]
Jun 03 23:33:20.096421 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:20.096405    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-pgqk6"
Jun 03 23:33:20.099341 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:20.099323    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-nswt5\""
Jun 03 23:33:20.124460 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:20.124429    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-pgqk6"]
Jun 03 23:33:20.202874 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:20.202839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18bc02fa-1874-436a-bce1-8653f487bc16-bound-sa-token\") pod \"cert-manager-59c77684cc-pgqk6\" (UID: \"18bc02fa-1874-436a-bce1-8653f487bc16\") " pod="cert-manager/cert-manager-59c77684cc-pgqk6"
Jun 03 23:33:20.203027 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:20.202882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r995\" (UniqueName: \"kubernetes.io/projected/18bc02fa-1874-436a-bce1-8653f487bc16-kube-api-access-2r995\") pod \"cert-manager-59c77684cc-pgqk6\" (UID: \"18bc02fa-1874-436a-bce1-8653f487bc16\") " pod="cert-manager/cert-manager-59c77684cc-pgqk6"
Jun 03 23:33:20.303561 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:20.303521    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18bc02fa-1874-436a-bce1-8653f487bc16-bound-sa-token\") pod \"cert-manager-59c77684cc-pgqk6\" (UID: \"18bc02fa-1874-436a-bce1-8653f487bc16\") " pod="cert-manager/cert-manager-59c77684cc-pgqk6"
Jun 03 23:33:20.303728 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:20.303573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r995\" (UniqueName: \"kubernetes.io/projected/18bc02fa-1874-436a-bce1-8653f487bc16-kube-api-access-2r995\") pod \"cert-manager-59c77684cc-pgqk6\" (UID: \"18bc02fa-1874-436a-bce1-8653f487bc16\") " pod="cert-manager/cert-manager-59c77684cc-pgqk6"
Jun 03 23:33:20.314241 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:20.314195    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r995\" (UniqueName: \"kubernetes.io/projected/18bc02fa-1874-436a-bce1-8653f487bc16-kube-api-access-2r995\") pod \"cert-manager-59c77684cc-pgqk6\" (UID: \"18bc02fa-1874-436a-bce1-8653f487bc16\") " pod="cert-manager/cert-manager-59c77684cc-pgqk6"
Jun 03 23:33:20.314353 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:20.314285    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18bc02fa-1874-436a-bce1-8653f487bc16-bound-sa-token\") pod \"cert-manager-59c77684cc-pgqk6\" (UID: \"18bc02fa-1874-436a-bce1-8653f487bc16\") " pod="cert-manager/cert-manager-59c77684cc-pgqk6"
Jun 03 23:33:20.410136 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:20.410050    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-pgqk6"
Jun 03 23:33:20.548667 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:20.548567    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-pgqk6"]
Jun 03 23:33:20.551281 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:33:20.551252    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18bc02fa_1874_436a_bce1_8653f487bc16.slice/crio-8a32c7113f5638eccac1e6e81cd3693eda9ebbbaff02973d33a011434b33940e WatchSource:0}: Error finding container 8a32c7113f5638eccac1e6e81cd3693eda9ebbbaff02973d33a011434b33940e: Status 404 returned error can't find the container with id 8a32c7113f5638eccac1e6e81cd3693eda9ebbbaff02973d33a011434b33940e
Jun 03 23:33:21.139925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:21.139887    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-pgqk6" event={"ID":"18bc02fa-1874-436a-bce1-8653f487bc16","Type":"ContainerStarted","Data":"a1d1acce2f040a3e894d149554ae28cb71b96dbb8dfb7157edb9d00274c61f1e"}
Jun 03 23:33:21.140323 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:21.139940    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-pgqk6" event={"ID":"18bc02fa-1874-436a-bce1-8653f487bc16","Type":"ContainerStarted","Data":"8a32c7113f5638eccac1e6e81cd3693eda9ebbbaff02973d33a011434b33940e"}
Jun 03 23:33:41.913916 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:41.913858    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-59c77684cc-pgqk6" podStartSLOduration=21.91384119 podStartE2EDuration="21.91384119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:33:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:33:21.186838778 +0000 UTC m=+523.349516853" watchObservedRunningTime="2026-06-03 23:33:41.91384119 +0000 UTC m=+544.076519282"
Jun 03 23:33:41.914425 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:41.913993    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"]
Jun 03 23:33:41.917868 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:41.917852    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:41.921340 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:41.921318    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jun 03 23:33:41.921464 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:41.921351    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4jsbw\""
Jun 03 23:33:41.921570 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:41.921553    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jun 03 23:33:41.922073 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:41.922058    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jun 03 23:33:41.926271 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:41.926250    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jun 03 23:33:41.950672 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:41.950640    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"]
Jun 03 23:33:41.970876 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:41.970850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/966fd4b6-a8b2-4b59-b62b-a723ee509b6d-webhook-cert\") pod \"opendatahub-operator-controller-manager-f87df8b57-lsfmk\" (UID: \"966fd4b6-a8b2-4b59-b62b-a723ee509b6d\") " pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:41.971013 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:41.970898    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/966fd4b6-a8b2-4b59-b62b-a723ee509b6d-tmp\") pod \"opendatahub-operator-controller-manager-f87df8b57-lsfmk\" (UID: \"966fd4b6-a8b2-4b59-b62b-a723ee509b6d\") " pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:41.971013 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:41.970967    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/966fd4b6-a8b2-4b59-b62b-a723ee509b6d-manifests\") pod \"opendatahub-operator-controller-manager-f87df8b57-lsfmk\" (UID: \"966fd4b6-a8b2-4b59-b62b-a723ee509b6d\") " pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:41.971180 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:41.971027    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/966fd4b6-a8b2-4b59-b62b-a723ee509b6d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f87df8b57-lsfmk\" (UID: \"966fd4b6-a8b2-4b59-b62b-a723ee509b6d\") " pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:41.971180 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:41.971065    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv6c2\" (UniqueName: \"kubernetes.io/projected/966fd4b6-a8b2-4b59-b62b-a723ee509b6d-kube-api-access-jv6c2\") pod \"opendatahub-operator-controller-manager-f87df8b57-lsfmk\" (UID: \"966fd4b6-a8b2-4b59-b62b-a723ee509b6d\") " pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:42.071891 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:42.071854    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/966fd4b6-a8b2-4b59-b62b-a723ee509b6d-manifests\") pod \"opendatahub-operator-controller-manager-f87df8b57-lsfmk\" (UID: \"966fd4b6-a8b2-4b59-b62b-a723ee509b6d\") " pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:42.072081 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:42.071901    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/966fd4b6-a8b2-4b59-b62b-a723ee509b6d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f87df8b57-lsfmk\" (UID: \"966fd4b6-a8b2-4b59-b62b-a723ee509b6d\") " pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:42.072081 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:42.071933    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv6c2\" (UniqueName: \"kubernetes.io/projected/966fd4b6-a8b2-4b59-b62b-a723ee509b6d-kube-api-access-jv6c2\") pod \"opendatahub-operator-controller-manager-f87df8b57-lsfmk\" (UID: \"966fd4b6-a8b2-4b59-b62b-a723ee509b6d\") " pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:42.072081 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:42.071960    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/966fd4b6-a8b2-4b59-b62b-a723ee509b6d-webhook-cert\") pod \"opendatahub-operator-controller-manager-f87df8b57-lsfmk\" (UID: \"966fd4b6-a8b2-4b59-b62b-a723ee509b6d\") " pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:42.072081 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:42.071995    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/966fd4b6-a8b2-4b59-b62b-a723ee509b6d-tmp\") pod \"opendatahub-operator-controller-manager-f87df8b57-lsfmk\" (UID: \"966fd4b6-a8b2-4b59-b62b-a723ee509b6d\") " pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:42.072327 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:42.072291    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/966fd4b6-a8b2-4b59-b62b-a723ee509b6d-manifests\") pod \"opendatahub-operator-controller-manager-f87df8b57-lsfmk\" (UID: \"966fd4b6-a8b2-4b59-b62b-a723ee509b6d\") " pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:42.074297 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:42.074270    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/966fd4b6-a8b2-4b59-b62b-a723ee509b6d-tmp\") pod \"opendatahub-operator-controller-manager-f87df8b57-lsfmk\" (UID: \"966fd4b6-a8b2-4b59-b62b-a723ee509b6d\") " pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:42.074522 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:42.074495    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/966fd4b6-a8b2-4b59-b62b-a723ee509b6d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f87df8b57-lsfmk\" (UID: \"966fd4b6-a8b2-4b59-b62b-a723ee509b6d\") " pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:42.074595 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:42.074581    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/966fd4b6-a8b2-4b59-b62b-a723ee509b6d-webhook-cert\") pod \"opendatahub-operator-controller-manager-f87df8b57-lsfmk\" (UID: \"966fd4b6-a8b2-4b59-b62b-a723ee509b6d\") " pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:42.084638 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:42.084618    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv6c2\" (UniqueName: \"kubernetes.io/projected/966fd4b6-a8b2-4b59-b62b-a723ee509b6d-kube-api-access-jv6c2\") pod \"opendatahub-operator-controller-manager-f87df8b57-lsfmk\" (UID: \"966fd4b6-a8b2-4b59-b62b-a723ee509b6d\") " pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:42.227410 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:42.227322    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:42.365297 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:42.365267    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"]
Jun 03 23:33:42.370359 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:33:42.370330    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod966fd4b6_a8b2_4b59_b62b_a723ee509b6d.slice/crio-7dfb93d1ba0704ea7af1944a4f41de10d40bd863fb84356e293745cf9a7df05f WatchSource:0}: Error finding container 7dfb93d1ba0704ea7af1944a4f41de10d40bd863fb84356e293745cf9a7df05f: Status 404 returned error can't find the container with id 7dfb93d1ba0704ea7af1944a4f41de10d40bd863fb84356e293745cf9a7df05f
Jun 03 23:33:43.223813 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:43.223767    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk" event={"ID":"966fd4b6-a8b2-4b59-b62b-a723ee509b6d","Type":"ContainerStarted","Data":"7dfb93d1ba0704ea7af1944a4f41de10d40bd863fb84356e293745cf9a7df05f"}
Jun 03 23:33:45.234624 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:45.234593    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk" event={"ID":"966fd4b6-a8b2-4b59-b62b-a723ee509b6d","Type":"ContainerStarted","Data":"98ccc59d2ad3f044cdfaa6395eff52fa3bd555ab142ab370c59a577bc2e46409"}
Jun 03 23:33:46.240692 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:46.240654    2577 generic.go:356] "Generic (PLEG): container finished" podID="966fd4b6-a8b2-4b59-b62b-a723ee509b6d" containerID="98ccc59d2ad3f044cdfaa6395eff52fa3bd555ab142ab370c59a577bc2e46409" exitCode=0
Jun 03 23:33:46.241072 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:46.240698    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk" event={"ID":"966fd4b6-a8b2-4b59-b62b-a723ee509b6d","Type":"ContainerDied","Data":"98ccc59d2ad3f044cdfaa6395eff52fa3bd555ab142ab370c59a577bc2e46409"}
Jun 03 23:33:47.249711 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:47.249675    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk" event={"ID":"966fd4b6-a8b2-4b59-b62b-a723ee509b6d","Type":"ContainerStarted","Data":"0288e90550ed17652cec653524a145a14abb5a8ccc8a2f37ae59d1325c910b1b"}
Jun 03 23:33:47.250125 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:47.249834    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:33:47.291334 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:47.291283    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk" podStartSLOduration=2.260923032 podStartE2EDuration="6.29126598s" totalImagesPullingTime="3.048786096s" totalInitContainerRuntime="981.556852ms" isStatefulPod=false podCreationTimestamp="2026-06-03 23:33:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:33:47.290774725 +0000 UTC m=+549.453452801" watchObservedRunningTime="2026-06-03 23:33:47.29126598 +0000 UTC m=+549.453944065"
Jun 03 23:33:58.256630 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:33:58.256601    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f87df8b57-lsfmk"
Jun 03 23:34:02.876573 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:02.876533    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-57684f659d-2qfpm"]
Jun 03 23:34:02.881517 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:02.881490    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57684f659d-2qfpm"
Jun 03 23:34:02.883893 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:02.883866    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9xnmf\""
Jun 03 23:34:02.883893 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:02.883877    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jun 03 23:34:02.883893 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:02.883883    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jun 03 23:34:02.891998 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:02.891977    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57684f659d-2qfpm"]
Jun 03 23:34:02.928043 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:02.928008    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2-tls-certs\") pod \"kube-auth-proxy-57684f659d-2qfpm\" (UID: \"3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2\") " pod="openshift-ingress/kube-auth-proxy-57684f659d-2qfpm"
Jun 03 23:34:02.928170 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:02.928137    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b7mh\" (UniqueName: \"kubernetes.io/projected/3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2-kube-api-access-9b7mh\") pod \"kube-auth-proxy-57684f659d-2qfpm\" (UID: \"3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2\") " pod="openshift-ingress/kube-auth-proxy-57684f659d-2qfpm"
Jun 03 23:34:02.928245 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:02.928166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2-tmp\") pod \"kube-auth-proxy-57684f659d-2qfpm\" (UID: \"3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2\") " pod="openshift-ingress/kube-auth-proxy-57684f659d-2qfpm"
Jun 03 23:34:03.029488 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:03.029441    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b7mh\" (UniqueName: \"kubernetes.io/projected/3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2-kube-api-access-9b7mh\") pod \"kube-auth-proxy-57684f659d-2qfpm\" (UID: \"3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2\") " pod="openshift-ingress/kube-auth-proxy-57684f659d-2qfpm"
Jun 03 23:34:03.029488 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:03.029490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2-tmp\") pod \"kube-auth-proxy-57684f659d-2qfpm\" (UID: \"3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2\") " pod="openshift-ingress/kube-auth-proxy-57684f659d-2qfpm"
Jun 03 23:34:03.029710 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:03.029526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2-tls-certs\") pod \"kube-auth-proxy-57684f659d-2qfpm\" (UID: \"3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2\") " pod="openshift-ingress/kube-auth-proxy-57684f659d-2qfpm"
Jun 03 23:34:03.029710 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:34:03.029620    2577 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jun 03 23:34:03.029710 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:34:03.029696    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2-tls-certs podName:3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2 nodeName:}" failed. No retries permitted until 2026-06-03 23:34:03.529681079 +0000 UTC m=+565.692359130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2-tls-certs") pod "kube-auth-proxy-57684f659d-2qfpm" (UID: "3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2") : secret "kube-auth-proxy-tls" not found
Jun 03 23:34:03.031874 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:03.031848    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2-tmp\") pod \"kube-auth-proxy-57684f659d-2qfpm\" (UID: \"3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2\") " pod="openshift-ingress/kube-auth-proxy-57684f659d-2qfpm"
Jun 03 23:34:03.039621 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:03.039592    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b7mh\" (UniqueName: \"kubernetes.io/projected/3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2-kube-api-access-9b7mh\") pod \"kube-auth-proxy-57684f659d-2qfpm\" (UID: \"3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2\") " pod="openshift-ingress/kube-auth-proxy-57684f659d-2qfpm"
Jun 03 23:34:03.533293 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:03.533260    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2-tls-certs\") pod \"kube-auth-proxy-57684f659d-2qfpm\" (UID: \"3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2\") " pod="openshift-ingress/kube-auth-proxy-57684f659d-2qfpm"
Jun 03 23:34:03.535656 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:03.535619    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2-tls-certs\") pod \"kube-auth-proxy-57684f659d-2qfpm\" (UID: \"3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2\") " pod="openshift-ingress/kube-auth-proxy-57684f659d-2qfpm"
Jun 03 23:34:03.791141 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:03.791057    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-57684f659d-2qfpm"
Jun 03 23:34:03.932504 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:03.928228    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-57684f659d-2qfpm"]
Jun 03 23:34:03.934155 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:34:03.934125    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e12d177_e8b3_4f63_a3d7_90c9ac28cfc2.slice/crio-f289b174b60ceb0db10aa6bd5fa1dcc7617e3f090d5f6a8ae2c233955fc2b054 WatchSource:0}: Error finding container f289b174b60ceb0db10aa6bd5fa1dcc7617e3f090d5f6a8ae2c233955fc2b054: Status 404 returned error can't find the container with id f289b174b60ceb0db10aa6bd5fa1dcc7617e3f090d5f6a8ae2c233955fc2b054
Jun 03 23:34:04.321516 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:04.321317    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57684f659d-2qfpm" event={"ID":"3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2","Type":"ContainerStarted","Data":"f289b174b60ceb0db10aa6bd5fa1dcc7617e3f090d5f6a8ae2c233955fc2b054"}
Jun 03 23:34:07.338049 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:07.338012    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-57684f659d-2qfpm" event={"ID":"3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2","Type":"ContainerStarted","Data":"b1be9fa4d2a246c4207fa093248f6a749fa47fded26d943f296d214b449517f9"}
Jun 03 23:34:07.358741 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:07.358678    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-57684f659d-2qfpm" podStartSLOduration=2.825986343 podStartE2EDuration="5.358657566s" totalImagesPullingTime="2.532671223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:34:07.357019211 +0000 UTC m=+569.519697310" watchObservedRunningTime="2026-06-03 23:34:07.358657566 +0000 UTC m=+569.521335641"
Jun 03 23:34:12.299860 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.299826    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"]
Jun 03 23:34:12.303459 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.303440    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"
Jun 03 23:34:12.306068 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.306046    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jun 03 23:34:12.307271 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.307083    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jun 03 23:34:12.307271 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.307106    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jun 03 23:34:12.307271 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.307188    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jun 03 23:34:12.307271 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.307227    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jun 03 23:34:12.307534 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.307295    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-khfml\""
Jun 03 23:34:12.309098 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.309075    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"]
Jun 03 23:34:12.413929 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.413897    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa22b043-e050-418b-8628-fc777eb4e306-cert\") pod \"lws-controller-manager-6845fb6b5-7rc4p\" (UID: \"aa22b043-e050-418b-8628-fc777eb4e306\") " pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"
Jun 03 23:34:12.414138 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.413954    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jpcc\" (UniqueName: \"kubernetes.io/projected/aa22b043-e050-418b-8628-fc777eb4e306-kube-api-access-6jpcc\") pod \"lws-controller-manager-6845fb6b5-7rc4p\" (UID: \"aa22b043-e050-418b-8628-fc777eb4e306\") " pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"
Jun 03 23:34:12.414138 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.414050    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa22b043-e050-418b-8628-fc777eb4e306-metrics-cert\") pod \"lws-controller-manager-6845fb6b5-7rc4p\" (UID: \"aa22b043-e050-418b-8628-fc777eb4e306\") " pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"
Jun 03 23:34:12.414138 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.414092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aa22b043-e050-418b-8628-fc777eb4e306-manager-config\") pod \"lws-controller-manager-6845fb6b5-7rc4p\" (UID: \"aa22b043-e050-418b-8628-fc777eb4e306\") " pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"
Jun 03 23:34:12.515386 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.515348    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jpcc\" (UniqueName: \"kubernetes.io/projected/aa22b043-e050-418b-8628-fc777eb4e306-kube-api-access-6jpcc\") pod \"lws-controller-manager-6845fb6b5-7rc4p\" (UID: \"aa22b043-e050-418b-8628-fc777eb4e306\") " pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"
Jun 03 23:34:12.515386 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.515392    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa22b043-e050-418b-8628-fc777eb4e306-metrics-cert\") pod \"lws-controller-manager-6845fb6b5-7rc4p\" (UID: \"aa22b043-e050-418b-8628-fc777eb4e306\") " pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"
Jun 03 23:34:12.515594 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.515507    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aa22b043-e050-418b-8628-fc777eb4e306-manager-config\") pod \"lws-controller-manager-6845fb6b5-7rc4p\" (UID: \"aa22b043-e050-418b-8628-fc777eb4e306\") " pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"
Jun 03 23:34:12.515594 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.515577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa22b043-e050-418b-8628-fc777eb4e306-cert\") pod \"lws-controller-manager-6845fb6b5-7rc4p\" (UID: \"aa22b043-e050-418b-8628-fc777eb4e306\") " pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"
Jun 03 23:34:12.516301 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.516279    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aa22b043-e050-418b-8628-fc777eb4e306-manager-config\") pod \"lws-controller-manager-6845fb6b5-7rc4p\" (UID: \"aa22b043-e050-418b-8628-fc777eb4e306\") " pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"
Jun 03 23:34:12.518036 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.518013    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa22b043-e050-418b-8628-fc777eb4e306-metrics-cert\") pod \"lws-controller-manager-6845fb6b5-7rc4p\" (UID: \"aa22b043-e050-418b-8628-fc777eb4e306\") " pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"
Jun 03 23:34:12.518118 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.518082    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa22b043-e050-418b-8628-fc777eb4e306-cert\") pod \"lws-controller-manager-6845fb6b5-7rc4p\" (UID: \"aa22b043-e050-418b-8628-fc777eb4e306\") " pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"
Jun 03 23:34:12.529616 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.529591    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jpcc\" (UniqueName: \"kubernetes.io/projected/aa22b043-e050-418b-8628-fc777eb4e306-kube-api-access-6jpcc\") pod \"lws-controller-manager-6845fb6b5-7rc4p\" (UID: \"aa22b043-e050-418b-8628-fc777eb4e306\") " pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"
Jun 03 23:34:12.614694 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.614660    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"
Jun 03 23:34:12.745865 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:12.745830    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"]
Jun 03 23:34:12.749726 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:34:12.749690    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa22b043_e050_418b_8628_fc777eb4e306.slice/crio-7ad9569634fbceda7299a4b2ffeaf2e6d76e792e02dc46c857114c779aa98883 WatchSource:0}: Error finding container 7ad9569634fbceda7299a4b2ffeaf2e6d76e792e02dc46c857114c779aa98883: Status 404 returned error can't find the container with id 7ad9569634fbceda7299a4b2ffeaf2e6d76e792e02dc46c857114c779aa98883
Jun 03 23:34:13.364535 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:13.364498    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p" event={"ID":"aa22b043-e050-418b-8628-fc777eb4e306","Type":"ContainerStarted","Data":"7ad9569634fbceda7299a4b2ffeaf2e6d76e792e02dc46c857114c779aa98883"}
Jun 03 23:34:15.374844 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:15.374806    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p" event={"ID":"aa22b043-e050-418b-8628-fc777eb4e306","Type":"ContainerStarted","Data":"1ff9a76df542f94ca3293a125edafbf9d833cea348e0997ef68003bd7cc83c01"}
Jun 03 23:34:15.375230 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:15.374869    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"
Jun 03 23:34:15.394733 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:15.394677    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p" podStartSLOduration=0.915637518 podStartE2EDuration="3.394661968s" totalImagesPullingTime="2.47902445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:34:15.39433802 +0000 UTC m=+577.557016095" watchObservedRunningTime="2026-06-03 23:34:15.394661968 +0000 UTC m=+577.557340041"
Jun 03 23:34:26.380921 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:26.380882    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6845fb6b5-7rc4p"
Jun 03 23:34:38.265480 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:38.265447    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 03 23:34:38.265931 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:34:38.265778    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 03 23:35:35.230258 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.230198    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-587cd6c99b-9q5cg"]
Jun 03 23:35:35.233350 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.233332    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.255075 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.255050    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-587cd6c99b-9q5cg"]
Jun 03 23:35:35.406897 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.406858    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84ec87ca-9194-42d0-a328-0c0e277389ff-console-serving-cert\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.407094 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.406976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84ec87ca-9194-42d0-a328-0c0e277389ff-console-config\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.407094 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.407003    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84ec87ca-9194-42d0-a328-0c0e277389ff-oauth-serving-cert\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.407094 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.407075    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84ec87ca-9194-42d0-a328-0c0e277389ff-console-oauth-config\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.407306 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.407161    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84ec87ca-9194-42d0-a328-0c0e277389ff-trusted-ca-bundle\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.407365 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.407304    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wwgp\" (UniqueName: \"kubernetes.io/projected/84ec87ca-9194-42d0-a328-0c0e277389ff-kube-api-access-4wwgp\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.407365 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.407342    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84ec87ca-9194-42d0-a328-0c0e277389ff-service-ca\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.507943 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.507859    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84ec87ca-9194-42d0-a328-0c0e277389ff-console-config\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.507943 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.507895    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84ec87ca-9194-42d0-a328-0c0e277389ff-oauth-serving-cert\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.508135 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.508021    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84ec87ca-9194-42d0-a328-0c0e277389ff-console-oauth-config\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.508135 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.508069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84ec87ca-9194-42d0-a328-0c0e277389ff-trusted-ca-bundle\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.508135 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.508102    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wwgp\" (UniqueName: \"kubernetes.io/projected/84ec87ca-9194-42d0-a328-0c0e277389ff-kube-api-access-4wwgp\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.508135 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.508125    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84ec87ca-9194-42d0-a328-0c0e277389ff-service-ca\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.508365 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.508149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84ec87ca-9194-42d0-a328-0c0e277389ff-console-serving-cert\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.508656 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.508628    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84ec87ca-9194-42d0-a328-0c0e277389ff-oauth-serving-cert\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.508761 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.508705    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84ec87ca-9194-42d0-a328-0c0e277389ff-console-config\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.508761 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.508750    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84ec87ca-9194-42d0-a328-0c0e277389ff-service-ca\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.509010 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.508989    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84ec87ca-9194-42d0-a328-0c0e277389ff-trusted-ca-bundle\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.510361 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.510341    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84ec87ca-9194-42d0-a328-0c0e277389ff-console-oauth-config\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.510465 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.510448    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84ec87ca-9194-42d0-a328-0c0e277389ff-console-serving-cert\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.516476 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.516459    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wwgp\" (UniqueName: \"kubernetes.io/projected/84ec87ca-9194-42d0-a328-0c0e277389ff-kube-api-access-4wwgp\") pod \"console-587cd6c99b-9q5cg\" (UID: \"84ec87ca-9194-42d0-a328-0c0e277389ff\") " pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.542352 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.542333    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:35.672948 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.672918    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-587cd6c99b-9q5cg"]
Jun 03 23:35:35.674858 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:35:35.674832    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84ec87ca_9194_42d0_a328_0c0e277389ff.slice/crio-55736235aa15ec6590961027dc8819e29257c3bef958ab5c26993dd1ecb1e325 WatchSource:0}: Error finding container 55736235aa15ec6590961027dc8819e29257c3bef958ab5c26993dd1ecb1e325: Status 404 returned error can't find the container with id 55736235aa15ec6590961027dc8819e29257c3bef958ab5c26993dd1ecb1e325
Jun 03 23:35:35.704335 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:35.704308    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-587cd6c99b-9q5cg" event={"ID":"84ec87ca-9194-42d0-a328-0c0e277389ff","Type":"ContainerStarted","Data":"55736235aa15ec6590961027dc8819e29257c3bef958ab5c26993dd1ecb1e325"}
Jun 03 23:35:36.710285 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:36.710241    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-587cd6c99b-9q5cg" event={"ID":"84ec87ca-9194-42d0-a328-0c0e277389ff","Type":"ContainerStarted","Data":"5a228307f52b43fee017c7d7d0f0b1c281afd9963e49fbe6a14a2481171203c5"}
Jun 03 23:35:36.732165 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:36.732115    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-587cd6c99b-9q5cg" podStartSLOduration=1.732100191 podStartE2EDuration="1.732100191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:35:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:35:36.730241438 +0000 UTC m=+658.892919509" watchObservedRunningTime="2026-06-03 23:35:36.732100191 +0000 UTC m=+658.894778264"
Jun 03 23:35:45.542698 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:45.542666    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:45.543147 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:45.542721    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:45.547482 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:45.547457    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:45.750239 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:45.750196    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-587cd6c99b-9q5cg"
Jun 03 23:35:45.811875 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:35:45.811801    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56bcd9f976-nvqrf"]
Jun 03 23:36:10.833073 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:10.833030    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-56bcd9f976-nvqrf" podUID="1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2" containerName="console" containerID="cri-o://82b2b653eb0fcf94404f7e044547bc6cb6c961719040ad6ddb1a8d836c9b949a" gracePeriod=15
Jun 03 23:36:11.087129 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.087073    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:36:11.195948 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.195912    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-config\") pod \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") "
Jun 03 23:36:11.195948 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.195947    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-oauth-config\") pod \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") "
Jun 03 23:36:11.196202 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.195987    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-oauth-serving-cert\") pod \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") "
Jun 03 23:36:11.196202 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.196013    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-trusted-ca-bundle\") pod \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") "
Jun 03 23:36:11.196349 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.196202    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-serving-cert\") pod \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") "
Jun 03 23:36:11.196349 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.196285    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-service-ca\") pod \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") "
Jun 03 23:36:11.196349 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.196330    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bt4fb\" (UniqueName: \"kubernetes.io/projected/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-kube-api-access-bt4fb\") pod \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\" (UID: \"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2\") "
Jun 03 23:36:11.196492 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.196418    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-config" (OuterVolumeSpecName: "console-config") pod "1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2" (UID: "1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:36:11.196492 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.196434    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2" (UID: "1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:36:11.196492 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.196468    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2" (UID: "1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:36:11.196699 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.196669    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-service-ca" (OuterVolumeSpecName: "service-ca") pod "1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2" (UID: "1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:36:11.196826 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.196746    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-service-ca\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:36:11.196826 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.196762    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-config\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:36:11.196826 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.196773    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-oauth-serving-cert\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:36:11.196826 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.196782    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-trusted-ca-bundle\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:36:11.198302 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.198269    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2" (UID: "1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 03 23:36:11.198382 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.198298    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-kube-api-access-bt4fb" (OuterVolumeSpecName: "kube-api-access-bt4fb") pod "1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2" (UID: "1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2"). InnerVolumeSpecName "kube-api-access-bt4fb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 23:36:11.198382 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.198355    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2" (UID: "1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 03 23:36:11.297534 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.297497    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-oauth-config\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:36:11.297534 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.297527    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-console-serving-cert\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:36:11.297534 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.297536    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bt4fb\" (UniqueName: \"kubernetes.io/projected/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2-kube-api-access-bt4fb\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:36:11.863428 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.863396    2577 generic.go:356] "Generic (PLEG): container finished" podID="1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2" containerID="82b2b653eb0fcf94404f7e044547bc6cb6c961719040ad6ddb1a8d836c9b949a" exitCode=0
Jun 03 23:36:11.863854 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.863436    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56bcd9f976-nvqrf" event={"ID":"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2","Type":"ContainerDied","Data":"82b2b653eb0fcf94404f7e044547bc6cb6c961719040ad6ddb1a8d836c9b949a"}
Jun 03 23:36:11.863854 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.863468    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56bcd9f976-nvqrf"
Jun 03 23:36:11.863854 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.863491    2577 scope.go:117] "RemoveContainer" containerID="82b2b653eb0fcf94404f7e044547bc6cb6c961719040ad6ddb1a8d836c9b949a"
Jun 03 23:36:11.863854 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.863481    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56bcd9f976-nvqrf" event={"ID":"1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2","Type":"ContainerDied","Data":"af36527414e283e189071c09d0591b710bef06adc87cd55257e6e7d04bd1ca83"}
Jun 03 23:36:11.872774 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.872553    2577 scope.go:117] "RemoveContainer" containerID="82b2b653eb0fcf94404f7e044547bc6cb6c961719040ad6ddb1a8d836c9b949a"
Jun 03 23:36:11.872975 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:36:11.872945    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82b2b653eb0fcf94404f7e044547bc6cb6c961719040ad6ddb1a8d836c9b949a\": container with ID starting with 82b2b653eb0fcf94404f7e044547bc6cb6c961719040ad6ddb1a8d836c9b949a not found: ID does not exist" containerID="82b2b653eb0fcf94404f7e044547bc6cb6c961719040ad6ddb1a8d836c9b949a"
Jun 03 23:36:11.873090 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.872985    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82b2b653eb0fcf94404f7e044547bc6cb6c961719040ad6ddb1a8d836c9b949a"} err="failed to get container status \"82b2b653eb0fcf94404f7e044547bc6cb6c961719040ad6ddb1a8d836c9b949a\": rpc error: code = NotFound desc = could not find container \"82b2b653eb0fcf94404f7e044547bc6cb6c961719040ad6ddb1a8d836c9b949a\": container with ID starting with 82b2b653eb0fcf94404f7e044547bc6cb6c961719040ad6ddb1a8d836c9b949a not found: ID does not exist"
Jun 03 23:36:11.885876 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.885850    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56bcd9f976-nvqrf"]
Jun 03 23:36:11.889744 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:11.889718    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56bcd9f976-nvqrf"]
Jun 03 23:36:12.278662 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:12.278587    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2" path="/var/lib/kubelet/pods/1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2/volumes"
Jun 03 23:36:41.862002 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:41.861963    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-gd2lj"]
Jun 03 23:36:41.862615 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:41.862426    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2" containerName="console"
Jun 03 23:36:41.862615 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:41.862446    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2" containerName="console"
Jun 03 23:36:41.862615 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:41.862559    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c058f12-b8e2-42c6-9ef0-0b1b98aa6ad2" containerName="console"
Jun 03 23:36:41.866506 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:41.866486    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gd2lj"
Jun 03 23:36:41.868985 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:41.868964    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jun 03 23:36:41.869102 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:41.869003    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jun 03 23:36:41.869626 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:41.869607    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-l59n2\""
Jun 03 23:36:41.871792 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:41.871773    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-gd2lj"]
Jun 03 23:36:41.962121 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:41.962089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2btwg\" (UniqueName: \"kubernetes.io/projected/e88722d4-e2d3-43f1-b074-8b870929806b-kube-api-access-2btwg\") pod \"authorino-58cd955675-gd2lj\" (UID: \"e88722d4-e2d3-43f1-b074-8b870929806b\") " pod="kuadrant-system/authorino-58cd955675-gd2lj"
Jun 03 23:36:42.017847 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:42.017817    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-dz6bj"]
Jun 03 23:36:42.021047 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:42.021030    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dz6bj"
Jun 03 23:36:42.028407 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:42.028380    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dz6bj"]
Jun 03 23:36:42.063143 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:42.063107    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2btwg\" (UniqueName: \"kubernetes.io/projected/e88722d4-e2d3-43f1-b074-8b870929806b-kube-api-access-2btwg\") pod \"authorino-58cd955675-gd2lj\" (UID: \"e88722d4-e2d3-43f1-b074-8b870929806b\") " pod="kuadrant-system/authorino-58cd955675-gd2lj"
Jun 03 23:36:42.063317 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:42.063176    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86mr4\" (UniqueName: \"kubernetes.io/projected/74848bec-2464-41b5-8f10-0a3f9f06cf33-kube-api-access-86mr4\") pod \"authorino-75d556bd77-dz6bj\" (UID: \"74848bec-2464-41b5-8f10-0a3f9f06cf33\") " pod="kuadrant-system/authorino-75d556bd77-dz6bj"
Jun 03 23:36:42.071472 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:42.071446    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2btwg\" (UniqueName: \"kubernetes.io/projected/e88722d4-e2d3-43f1-b074-8b870929806b-kube-api-access-2btwg\") pod \"authorino-58cd955675-gd2lj\" (UID: \"e88722d4-e2d3-43f1-b074-8b870929806b\") " pod="kuadrant-system/authorino-58cd955675-gd2lj"
Jun 03 23:36:42.163978 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:42.163890    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86mr4\" (UniqueName: \"kubernetes.io/projected/74848bec-2464-41b5-8f10-0a3f9f06cf33-kube-api-access-86mr4\") pod \"authorino-75d556bd77-dz6bj\" (UID: \"74848bec-2464-41b5-8f10-0a3f9f06cf33\") " pod="kuadrant-system/authorino-75d556bd77-dz6bj"
Jun 03 23:36:42.172142 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:42.172111    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86mr4\" (UniqueName: \"kubernetes.io/projected/74848bec-2464-41b5-8f10-0a3f9f06cf33-kube-api-access-86mr4\") pod \"authorino-75d556bd77-dz6bj\" (UID: \"74848bec-2464-41b5-8f10-0a3f9f06cf33\") " pod="kuadrant-system/authorino-75d556bd77-dz6bj"
Jun 03 23:36:42.176829 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:42.176813    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gd2lj"
Jun 03 23:36:42.304335 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:42.304310    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-gd2lj"]
Jun 03 23:36:42.306706 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:36:42.306677    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode88722d4_e2d3_43f1_b074_8b870929806b.slice/crio-90944a100949f068d6d81fbc84ea467cf8df7cf55cb2513783c95f367376b9e6 WatchSource:0}: Error finding container 90944a100949f068d6d81fbc84ea467cf8df7cf55cb2513783c95f367376b9e6: Status 404 returned error can't find the container with id 90944a100949f068d6d81fbc84ea467cf8df7cf55cb2513783c95f367376b9e6
Jun 03 23:36:42.335518 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:42.335491    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dz6bj"
Jun 03 23:36:42.464417 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:42.464385    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dz6bj"]
Jun 03 23:36:42.466657 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:36:42.466624    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74848bec_2464_41b5_8f10_0a3f9f06cf33.slice/crio-007648d16066e9d20ff4f987d2095e2959d7b6e8e84d82aa171fb3a6ed5a48ff WatchSource:0}: Error finding container 007648d16066e9d20ff4f987d2095e2959d7b6e8e84d82aa171fb3a6ed5a48ff: Status 404 returned error can't find the container with id 007648d16066e9d20ff4f987d2095e2959d7b6e8e84d82aa171fb3a6ed5a48ff
Jun 03 23:36:43.001960 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:43.001919    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dz6bj" event={"ID":"74848bec-2464-41b5-8f10-0a3f9f06cf33","Type":"ContainerStarted","Data":"007648d16066e9d20ff4f987d2095e2959d7b6e8e84d82aa171fb3a6ed5a48ff"}
Jun 03 23:36:43.005700 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:43.005636    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gd2lj" event={"ID":"e88722d4-e2d3-43f1-b074-8b870929806b","Type":"ContainerStarted","Data":"90944a100949f068d6d81fbc84ea467cf8df7cf55cb2513783c95f367376b9e6"}
Jun 03 23:36:46.022552 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:46.022518    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gd2lj" event={"ID":"e88722d4-e2d3-43f1-b074-8b870929806b","Type":"ContainerStarted","Data":"91c397765fc42d33228721adc80d7b46992b0f96c8c6ce8d64327049fba94326"}
Jun 03 23:36:46.023923 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:46.023901    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dz6bj" event={"ID":"74848bec-2464-41b5-8f10-0a3f9f06cf33","Type":"ContainerStarted","Data":"d4928f064fd8b39b16c1cabe9cd4e725c57565b781a7e6676c95d1342226bf44"}
Jun 03 23:36:46.046117 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:46.046075    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-gd2lj" podStartSLOduration=1.819413897 podStartE2EDuration="5.046062217s" totalImagesPullingTime="3.22664832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:36:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:36:46.045246754 +0000 UTC m=+728.207924828" watchObservedRunningTime="2026-06-03 23:36:46.046062217 +0000 UTC m=+728.208740295"
Jun 03 23:36:46.067576 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:46.067527    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-dz6bj" podStartSLOduration=0.989639861 podStartE2EDuration="4.067512349s" totalImagesPullingTime="3.077872488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:36:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:36:46.065840544 +0000 UTC m=+728.228518618" watchObservedRunningTime="2026-06-03 23:36:46.067512349 +0000 UTC m=+728.230190427"
Jun 03 23:36:46.087094 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:46.087068    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-gd2lj"]
Jun 03 23:36:48.032692 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:48.032633    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-gd2lj" podUID="e88722d4-e2d3-43f1-b074-8b870929806b" containerName="authorino" containerID="cri-o://91c397765fc42d33228721adc80d7b46992b0f96c8c6ce8d64327049fba94326" gracePeriod=30
Jun 03 23:36:48.278028 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:48.278004    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gd2lj"
Jun 03 23:36:48.316778 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:48.316746    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2btwg\" (UniqueName: \"kubernetes.io/projected/e88722d4-e2d3-43f1-b074-8b870929806b-kube-api-access-2btwg\") pod \"e88722d4-e2d3-43f1-b074-8b870929806b\" (UID: \"e88722d4-e2d3-43f1-b074-8b870929806b\") "
Jun 03 23:36:48.318829 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:48.318803    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e88722d4-e2d3-43f1-b074-8b870929806b-kube-api-access-2btwg" (OuterVolumeSpecName: "kube-api-access-2btwg") pod "e88722d4-e2d3-43f1-b074-8b870929806b" (UID: "e88722d4-e2d3-43f1-b074-8b870929806b"). InnerVolumeSpecName "kube-api-access-2btwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 23:36:48.418348 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:48.418315    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2btwg\" (UniqueName: \"kubernetes.io/projected/e88722d4-e2d3-43f1-b074-8b870929806b-kube-api-access-2btwg\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:36:49.038030 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:49.037995    2577 generic.go:356] "Generic (PLEG): container finished" podID="e88722d4-e2d3-43f1-b074-8b870929806b" containerID="91c397765fc42d33228721adc80d7b46992b0f96c8c6ce8d64327049fba94326" exitCode=0
Jun 03 23:36:49.038487 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:49.038037    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-gd2lj"
Jun 03 23:36:49.038487 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:49.038065    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gd2lj" event={"ID":"e88722d4-e2d3-43f1-b074-8b870929806b","Type":"ContainerDied","Data":"91c397765fc42d33228721adc80d7b46992b0f96c8c6ce8d64327049fba94326"}
Jun 03 23:36:49.038487 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:49.038097    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-gd2lj" event={"ID":"e88722d4-e2d3-43f1-b074-8b870929806b","Type":"ContainerDied","Data":"90944a100949f068d6d81fbc84ea467cf8df7cf55cb2513783c95f367376b9e6"}
Jun 03 23:36:49.038487 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:49.038113    2577 scope.go:117] "RemoveContainer" containerID="91c397765fc42d33228721adc80d7b46992b0f96c8c6ce8d64327049fba94326"
Jun 03 23:36:49.046912 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:49.046896    2577 scope.go:117] "RemoveContainer" containerID="91c397765fc42d33228721adc80d7b46992b0f96c8c6ce8d64327049fba94326"
Jun 03 23:36:49.047113 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:36:49.047096    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91c397765fc42d33228721adc80d7b46992b0f96c8c6ce8d64327049fba94326\": container with ID starting with 91c397765fc42d33228721adc80d7b46992b0f96c8c6ce8d64327049fba94326 not found: ID does not exist" containerID="91c397765fc42d33228721adc80d7b46992b0f96c8c6ce8d64327049fba94326"
Jun 03 23:36:49.047176 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:49.047123    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91c397765fc42d33228721adc80d7b46992b0f96c8c6ce8d64327049fba94326"} err="failed to get container status \"91c397765fc42d33228721adc80d7b46992b0f96c8c6ce8d64327049fba94326\": rpc error: code = NotFound desc = could not find container \"91c397765fc42d33228721adc80d7b46992b0f96c8c6ce8d64327049fba94326\": container with ID starting with 91c397765fc42d33228721adc80d7b46992b0f96c8c6ce8d64327049fba94326 not found: ID does not exist"
Jun 03 23:36:49.056927 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:49.056908    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-gd2lj"]
Jun 03 23:36:49.060444 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:49.060426    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-gd2lj"]
Jun 03 23:36:50.278230 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:36:50.278182    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e88722d4-e2d3-43f1-b074-8b870929806b" path="/var/lib/kubelet/pods/e88722d4-e2d3-43f1-b074-8b870929806b/volumes"
Jun 03 23:37:35.580846 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:35.580808    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 23:37:35.581339 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:35.581236    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e88722d4-e2d3-43f1-b074-8b870929806b" containerName="authorino"
Jun 03 23:37:35.581339 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:35.581256    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e88722d4-e2d3-43f1-b074-8b870929806b" containerName="authorino"
Jun 03 23:37:35.581429 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:35.581341    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e88722d4-e2d3-43f1-b074-8b870929806b" containerName="authorino"
Jun 03 23:37:35.583544 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:35.583526    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 03 23:37:35.585751 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:35.585723    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\""
Jun 03 23:37:35.585853 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:35.585789    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\""
Jun 03 23:37:35.586448 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:35.586420    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-v9dhl\""
Jun 03 23:37:35.586561 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:35.586447    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\""
Jun 03 23:37:35.597614 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:35.597593    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 23:37:35.700528 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:35.700497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rxt5\" (UniqueName: \"kubernetes.io/projected/96b06857-3261-429d-937e-0c8cc45457bb-kube-api-access-2rxt5\") pod \"maas-keycloak-0\" (UID: \"96b06857-3261-429d-937e-0c8cc45457bb\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 23:37:35.801527 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:35.801496    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rxt5\" (UniqueName: \"kubernetes.io/projected/96b06857-3261-429d-937e-0c8cc45457bb-kube-api-access-2rxt5\") pod \"maas-keycloak-0\" (UID: \"96b06857-3261-429d-937e-0c8cc45457bb\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 23:37:35.810104 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:35.810077    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rxt5\" (UniqueName: \"kubernetes.io/projected/96b06857-3261-429d-937e-0c8cc45457bb-kube-api-access-2rxt5\") pod \"maas-keycloak-0\" (UID: \"96b06857-3261-429d-937e-0c8cc45457bb\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 23:37:35.892630 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:35.892560    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 03 23:37:36.021430 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:36.021397    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 23:37:36.025070 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:37:36.025037    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b06857_3261_429d_937e_0c8cc45457bb.slice/crio-38d38544e3f9e99590032f6e73bd00e4cb5e86f1f76633630a4fe05db158d0f6 WatchSource:0}: Error finding container 38d38544e3f9e99590032f6e73bd00e4cb5e86f1f76633630a4fe05db158d0f6: Status 404 returned error can't find the container with id 38d38544e3f9e99590032f6e73bd00e4cb5e86f1f76633630a4fe05db158d0f6
Jun 03 23:37:36.235828 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:36.235741    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"96b06857-3261-429d-937e-0c8cc45457bb","Type":"ContainerStarted","Data":"38d38544e3f9e99590032f6e73bd00e4cb5e86f1f76633630a4fe05db158d0f6"}
Jun 03 23:37:41.397802 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:41.397777    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\""
Jun 03 23:37:42.278001 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:42.277964    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"96b06857-3261-429d-937e-0c8cc45457bb","Type":"ContainerStarted","Data":"66218abc23227e91b1e8dbd71bb6f62c19fcaaa4eec41c06c087f4a98c451ba3"}
Jun 03 23:37:42.292570 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:42.292510    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.923500392 podStartE2EDuration="7.292490381s" totalImagesPullingTime="5.368989989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:37:42.290263957 +0000 UTC m=+784.452942034" watchObservedRunningTime="2026-06-03 23:37:42.292490381 +0000 UTC m=+784.455168458"
Jun 03 23:37:42.893644 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:42.893595    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0"
Jun 03 23:37:42.895707 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:42.895666    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.32:9000/health/started\": dial tcp 10.134.0.32:9000: connect: connection refused"
Jun 03 23:37:43.893248 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:43.893176    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.32:9000/health/started\": dial tcp 10.134.0.32:9000: connect: connection refused"
Jun 03 23:37:44.893582 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:44.893526    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.32:9000/health/started\": dial tcp 10.134.0.32:9000: connect: connection refused"
Jun 03 23:37:45.893511 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:45.892800    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0"
Jun 03 23:37:45.893511 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:45.893114    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.32:9000/health/started\": dial tcp 10.134.0.32:9000: connect: connection refused"
Jun 03 23:37:46.893595 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:46.893167    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.32:9000/health/started\": dial tcp 10.134.0.32:9000: connect: connection refused"
Jun 03 23:37:47.893633 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:47.893575    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.32:9000/health/started\": dial tcp 10.134.0.32:9000: connect: connection refused"
Jun 03 23:37:48.894065 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:48.894008    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.32:9000/health/started\": dial tcp 10.134.0.32:9000: connect: connection refused"
Jun 03 23:37:49.892997 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:49.892946    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.32:9000/health/started\": dial tcp 10.134.0.32:9000: connect: connection refused"
Jun 03 23:37:50.893383 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:50.893287    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.32:9000/health/started\": dial tcp 10.134.0.32:9000: connect: connection refused"
Jun 03 23:37:51.893855 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:51.893803    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.32:9000/health/started\": dial tcp 10.134.0.32:9000: connect: connection refused"
Jun 03 23:37:52.893884 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:52.893835    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.32:9000/health/started\": dial tcp 10.134.0.32:9000: connect: connection refused"
Jun 03 23:37:53.893272 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:53.893190    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.32:9000/health/started\": dial tcp 10.134.0.32:9000: connect: connection refused"
Jun 03 23:37:55.018875 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:55.018822    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0"
Jun 03 23:37:55.037020 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:37:55.036953    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 03 23:38:05.025801 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:05.025769    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0"
Jun 03 23:38:05.882080 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:05.882045    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dlnvb"]
Jun 03 23:38:05.885326 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:05.885306    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dlnvb"
Jun 03 23:38:05.892410 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:05.892389    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dlnvb"]
Jun 03 23:38:05.981122 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:05.981089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcm8w\" (UniqueName: \"kubernetes.io/projected/0d451586-8619-4745-8b7a-9836032b778e-kube-api-access-rcm8w\") pod \"authorino-5d4b67c446-dlnvb\" (UID: \"0d451586-8619-4745-8b7a-9836032b778e\") " pod="kuadrant-system/authorino-5d4b67c446-dlnvb"
Jun 03 23:38:06.081706 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.081671    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcm8w\" (UniqueName: \"kubernetes.io/projected/0d451586-8619-4745-8b7a-9836032b778e-kube-api-access-rcm8w\") pod \"authorino-5d4b67c446-dlnvb\" (UID: \"0d451586-8619-4745-8b7a-9836032b778e\") " pod="kuadrant-system/authorino-5d4b67c446-dlnvb"
Jun 03 23:38:06.089956 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.089925    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcm8w\" (UniqueName: \"kubernetes.io/projected/0d451586-8619-4745-8b7a-9836032b778e-kube-api-access-rcm8w\") pod \"authorino-5d4b67c446-dlnvb\" (UID: \"0d451586-8619-4745-8b7a-9836032b778e\") " pod="kuadrant-system/authorino-5d4b67c446-dlnvb"
Jun 03 23:38:06.117701 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.117675    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dlnvb"]
Jun 03 23:38:06.117877 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.117865    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dlnvb"
Jun 03 23:38:06.143624 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.143572    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c9597c988-tft6d"]
Jun 03 23:38:06.146801 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.146784    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c9597c988-tft6d"
Jun 03 23:38:06.157919 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.157899    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c9597c988-tft6d"]
Jun 03 23:38:06.182373 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.182348    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq6pb\" (UniqueName: \"kubernetes.io/projected/5167ee31-161f-435c-b844-bc97a472e208-kube-api-access-tq6pb\") pod \"authorino-c9597c988-tft6d\" (UID: \"5167ee31-161f-435c-b844-bc97a472e208\") " pod="kuadrant-system/authorino-c9597c988-tft6d"
Jun 03 23:38:06.183317 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.183295    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c9597c988-tft6d"]
Jun 03 23:38:06.183542 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:38:06.183521    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-tq6pb], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-c9597c988-tft6d" podUID="5167ee31-161f-435c-b844-bc97a472e208"
Jun 03 23:38:06.214403 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.214379    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-64fc754b9b-2mx2t"]
Jun 03 23:38:06.229178 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.228904    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64fc754b9b-2mx2t"]
Jun 03 23:38:06.229178 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.229044    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64fc754b9b-2mx2t"
Jun 03 23:38:06.231485 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.231459    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jun 03 23:38:06.255249 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.255228    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dlnvb"]
Jun 03 23:38:06.257626 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:38:06.257604    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d451586_8619_4745_8b7a_9836032b778e.slice/crio-8b8efd824769bb97d220808bbe8bda9bb24bf6ca45b1b4d6a950da87f0fbd347 WatchSource:0}: Error finding container 8b8efd824769bb97d220808bbe8bda9bb24bf6ca45b1b4d6a950da87f0fbd347: Status 404 returned error can't find the container with id 8b8efd824769bb97d220808bbe8bda9bb24bf6ca45b1b4d6a950da87f0fbd347
Jun 03 23:38:06.259060 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.259038    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 03 23:38:06.284015 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.283992    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq6pb\" (UniqueName: \"kubernetes.io/projected/5167ee31-161f-435c-b844-bc97a472e208-kube-api-access-tq6pb\") pod \"authorino-c9597c988-tft6d\" (UID: \"5167ee31-161f-435c-b844-bc97a472e208\") " pod="kuadrant-system/authorino-c9597c988-tft6d"
Jun 03 23:38:06.284130 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.284113    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhzdb\" (UniqueName: \"kubernetes.io/projected/984592c6-c108-4a27-aa8a-0b78d2c33bf2-kube-api-access-xhzdb\") pod \"authorino-64fc754b9b-2mx2t\" (UID: \"984592c6-c108-4a27-aa8a-0b78d2c33bf2\") " pod="kuadrant-system/authorino-64fc754b9b-2mx2t"
Jun 03 23:38:06.284196 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.284151    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/984592c6-c108-4a27-aa8a-0b78d2c33bf2-tls-cert\") pod \"authorino-64fc754b9b-2mx2t\" (UID: \"984592c6-c108-4a27-aa8a-0b78d2c33bf2\") " pod="kuadrant-system/authorino-64fc754b9b-2mx2t"
Jun 03 23:38:06.291600 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.291582    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq6pb\" (UniqueName: \"kubernetes.io/projected/5167ee31-161f-435c-b844-bc97a472e208-kube-api-access-tq6pb\") pod \"authorino-c9597c988-tft6d\" (UID: \"5167ee31-161f-435c-b844-bc97a472e208\") " pod="kuadrant-system/authorino-c9597c988-tft6d"
Jun 03 23:38:06.384557 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.384529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhzdb\" (UniqueName: \"kubernetes.io/projected/984592c6-c108-4a27-aa8a-0b78d2c33bf2-kube-api-access-xhzdb\") pod \"authorino-64fc754b9b-2mx2t\" (UID: \"984592c6-c108-4a27-aa8a-0b78d2c33bf2\") " pod="kuadrant-system/authorino-64fc754b9b-2mx2t"
Jun 03 23:38:06.384710 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.384562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/984592c6-c108-4a27-aa8a-0b78d2c33bf2-tls-cert\") pod \"authorino-64fc754b9b-2mx2t\" (UID: \"984592c6-c108-4a27-aa8a-0b78d2c33bf2\") " pod="kuadrant-system/authorino-64fc754b9b-2mx2t"
Jun 03 23:38:06.387442 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.387413    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/984592c6-c108-4a27-aa8a-0b78d2c33bf2-tls-cert\") pod \"authorino-64fc754b9b-2mx2t\" (UID: \"984592c6-c108-4a27-aa8a-0b78d2c33bf2\") " pod="kuadrant-system/authorino-64fc754b9b-2mx2t"
Jun 03 23:38:06.392529 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.392504    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhzdb\" (UniqueName: \"kubernetes.io/projected/984592c6-c108-4a27-aa8a-0b78d2c33bf2-kube-api-access-xhzdb\") pod \"authorino-64fc754b9b-2mx2t\" (UID: \"984592c6-c108-4a27-aa8a-0b78d2c33bf2\") " pod="kuadrant-system/authorino-64fc754b9b-2mx2t"
Jun 03 23:38:06.404163 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.404111    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-dlnvb" event={"ID":"0d451586-8619-4745-8b7a-9836032b778e","Type":"ContainerStarted","Data":"8b8efd824769bb97d220808bbe8bda9bb24bf6ca45b1b4d6a950da87f0fbd347"}
Jun 03 23:38:06.404163 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.404144    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c9597c988-tft6d"
Jun 03 23:38:06.411966 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.411947    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c9597c988-tft6d"
Jun 03 23:38:06.484870 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.484841    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tq6pb\" (UniqueName: \"kubernetes.io/projected/5167ee31-161f-435c-b844-bc97a472e208-kube-api-access-tq6pb\") pod \"5167ee31-161f-435c-b844-bc97a472e208\" (UID: \"5167ee31-161f-435c-b844-bc97a472e208\") "
Jun 03 23:38:06.486895 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.486864    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5167ee31-161f-435c-b844-bc97a472e208-kube-api-access-tq6pb" (OuterVolumeSpecName: "kube-api-access-tq6pb") pod "5167ee31-161f-435c-b844-bc97a472e208" (UID: "5167ee31-161f-435c-b844-bc97a472e208"). InnerVolumeSpecName "kube-api-access-tq6pb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 23:38:06.540341 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.540295    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64fc754b9b-2mx2t"
Jun 03 23:38:06.585489 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.585456    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tq6pb\" (UniqueName: \"kubernetes.io/projected/5167ee31-161f-435c-b844-bc97a472e208-kube-api-access-tq6pb\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:38:06.665750 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:06.665726    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64fc754b9b-2mx2t"]
Jun 03 23:38:06.668085 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:38:06.668060    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod984592c6_c108_4a27_aa8a_0b78d2c33bf2.slice/crio-35ececf6c75f536eb1c9dd557524ab8d2ae94fad518a535945ac7278e1cf9851 WatchSource:0}: Error finding container 35ececf6c75f536eb1c9dd557524ab8d2ae94fad518a535945ac7278e1cf9851: Status 404 returned error can't find the container with id 35ececf6c75f536eb1c9dd557524ab8d2ae94fad518a535945ac7278e1cf9851
Jun 03 23:38:07.410127 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.410087    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64fc754b9b-2mx2t" event={"ID":"984592c6-c108-4a27-aa8a-0b78d2c33bf2","Type":"ContainerStarted","Data":"0efad4d7f92d1ec1e32f474e814f3a1324b153a81aa1ad9e5a94ffd2f232b6ec"}
Jun 03 23:38:07.410598 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.410137    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64fc754b9b-2mx2t" event={"ID":"984592c6-c108-4a27-aa8a-0b78d2c33bf2","Type":"ContainerStarted","Data":"35ececf6c75f536eb1c9dd557524ab8d2ae94fad518a535945ac7278e1cf9851"}
Jun 03 23:38:07.411503 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.411473    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-dlnvb" event={"ID":"0d451586-8619-4745-8b7a-9836032b778e","Type":"ContainerStarted","Data":"ad73e2893503273a81b9e8378846b9e9c0e97aa12ccdc3efb4959dd8eacc0e63"}
Jun 03 23:38:07.411503 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.411497    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c9597c988-tft6d"
Jun 03 23:38:07.411682 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.411517    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-dlnvb" podUID="0d451586-8619-4745-8b7a-9836032b778e" containerName="authorino" containerID="cri-o://ad73e2893503273a81b9e8378846b9e9c0e97aa12ccdc3efb4959dd8eacc0e63" gracePeriod=30
Jun 03 23:38:07.427155 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.427115    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-64fc754b9b-2mx2t" podStartSLOduration=1.095916617 podStartE2EDuration="1.427100546s" totalImagesPullingTime="331.183929ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:38:07.425582253 +0000 UTC m=+809.588260327" watchObservedRunningTime="2026-06-03 23:38:07.427100546 +0000 UTC m=+809.589778631"
Jun 03 23:38:07.449756 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.449729    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c9597c988-tft6d"]
Jun 03 23:38:07.455904 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.455881    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dz6bj"]
Jun 03 23:38:07.456062 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.456042    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-dz6bj" podUID="74848bec-2464-41b5-8f10-0a3f9f06cf33" containerName="authorino" containerID="cri-o://d4928f064fd8b39b16c1cabe9cd4e725c57565b781a7e6676c95d1342226bf44" gracePeriod=30
Jun 03 23:38:07.459275 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.459249    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c9597c988-tft6d"]
Jun 03 23:38:07.476222 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.476156    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-dlnvb" podStartSLOduration=2.082760203 podStartE2EDuration="2.47613698s" totalImagesPullingTime="393.376777ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:38:07.474005498 +0000 UTC m=+809.636683574" watchObservedRunningTime="2026-06-03 23:38:07.47613698 +0000 UTC m=+809.638815071"
Jun 03 23:38:07.714592 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.714565    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dlnvb"
Jun 03 23:38:07.717258 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.717239    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dz6bj"
Jun 03 23:38:07.795581 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.795547    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86mr4\" (UniqueName: \"kubernetes.io/projected/74848bec-2464-41b5-8f10-0a3f9f06cf33-kube-api-access-86mr4\") pod \"74848bec-2464-41b5-8f10-0a3f9f06cf33\" (UID: \"74848bec-2464-41b5-8f10-0a3f9f06cf33\") "
Jun 03 23:38:07.795707 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.795616    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcm8w\" (UniqueName: \"kubernetes.io/projected/0d451586-8619-4745-8b7a-9836032b778e-kube-api-access-rcm8w\") pod \"0d451586-8619-4745-8b7a-9836032b778e\" (UID: \"0d451586-8619-4745-8b7a-9836032b778e\") "
Jun 03 23:38:07.797601 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.797575    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74848bec-2464-41b5-8f10-0a3f9f06cf33-kube-api-access-86mr4" (OuterVolumeSpecName: "kube-api-access-86mr4") pod "74848bec-2464-41b5-8f10-0a3f9f06cf33" (UID: "74848bec-2464-41b5-8f10-0a3f9f06cf33"). InnerVolumeSpecName "kube-api-access-86mr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 23:38:07.797673 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.797640    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d451586-8619-4745-8b7a-9836032b778e-kube-api-access-rcm8w" (OuterVolumeSpecName: "kube-api-access-rcm8w") pod "0d451586-8619-4745-8b7a-9836032b778e" (UID: "0d451586-8619-4745-8b7a-9836032b778e"). InnerVolumeSpecName "kube-api-access-rcm8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 23:38:07.896053 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.896029    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86mr4\" (UniqueName: \"kubernetes.io/projected/74848bec-2464-41b5-8f10-0a3f9f06cf33-kube-api-access-86mr4\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:38:07.896053 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:07.896050    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcm8w\" (UniqueName: \"kubernetes.io/projected/0d451586-8619-4745-8b7a-9836032b778e-kube-api-access-rcm8w\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:38:08.278132 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.278103    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5167ee31-161f-435c-b844-bc97a472e208" path="/var/lib/kubelet/pods/5167ee31-161f-435c-b844-bc97a472e208/volumes"
Jun 03 23:38:08.416303 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.416223    2577 generic.go:356] "Generic (PLEG): container finished" podID="74848bec-2464-41b5-8f10-0a3f9f06cf33" containerID="d4928f064fd8b39b16c1cabe9cd4e725c57565b781a7e6676c95d1342226bf44" exitCode=0
Jun 03 23:38:08.416303 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.416255    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dz6bj"
Jun 03 23:38:08.416759 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.416310    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dz6bj" event={"ID":"74848bec-2464-41b5-8f10-0a3f9f06cf33","Type":"ContainerDied","Data":"d4928f064fd8b39b16c1cabe9cd4e725c57565b781a7e6676c95d1342226bf44"}
Jun 03 23:38:08.416759 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.416348    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dz6bj" event={"ID":"74848bec-2464-41b5-8f10-0a3f9f06cf33","Type":"ContainerDied","Data":"007648d16066e9d20ff4f987d2095e2959d7b6e8e84d82aa171fb3a6ed5a48ff"}
Jun 03 23:38:08.416759 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.416367    2577 scope.go:117] "RemoveContainer" containerID="d4928f064fd8b39b16c1cabe9cd4e725c57565b781a7e6676c95d1342226bf44"
Jun 03 23:38:08.417705 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.417637    2577 generic.go:356] "Generic (PLEG): container finished" podID="0d451586-8619-4745-8b7a-9836032b778e" containerID="ad73e2893503273a81b9e8378846b9e9c0e97aa12ccdc3efb4959dd8eacc0e63" exitCode=0
Jun 03 23:38:08.417705 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.417672    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dlnvb"
Jun 03 23:38:08.417803 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.417731    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-dlnvb" event={"ID":"0d451586-8619-4745-8b7a-9836032b778e","Type":"ContainerDied","Data":"ad73e2893503273a81b9e8378846b9e9c0e97aa12ccdc3efb4959dd8eacc0e63"}
Jun 03 23:38:08.417803 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.417768    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-dlnvb" event={"ID":"0d451586-8619-4745-8b7a-9836032b778e","Type":"ContainerDied","Data":"8b8efd824769bb97d220808bbe8bda9bb24bf6ca45b1b4d6a950da87f0fbd347"}
Jun 03 23:38:08.425375 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.425360    2577 scope.go:117] "RemoveContainer" containerID="d4928f064fd8b39b16c1cabe9cd4e725c57565b781a7e6676c95d1342226bf44"
Jun 03 23:38:08.425638 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:38:08.425617    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4928f064fd8b39b16c1cabe9cd4e725c57565b781a7e6676c95d1342226bf44\": container with ID starting with d4928f064fd8b39b16c1cabe9cd4e725c57565b781a7e6676c95d1342226bf44 not found: ID does not exist" containerID="d4928f064fd8b39b16c1cabe9cd4e725c57565b781a7e6676c95d1342226bf44"
Jun 03 23:38:08.425693 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.425647    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4928f064fd8b39b16c1cabe9cd4e725c57565b781a7e6676c95d1342226bf44"} err="failed to get container status \"d4928f064fd8b39b16c1cabe9cd4e725c57565b781a7e6676c95d1342226bf44\": rpc error: code = NotFound desc = could not find container \"d4928f064fd8b39b16c1cabe9cd4e725c57565b781a7e6676c95d1342226bf44\": container with ID starting with d4928f064fd8b39b16c1cabe9cd4e725c57565b781a7e6676c95d1342226bf44 not found: ID does not exist"
Jun 03 23:38:08.425693 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.425667    2577 scope.go:117] "RemoveContainer" containerID="ad73e2893503273a81b9e8378846b9e9c0e97aa12ccdc3efb4959dd8eacc0e63"
Jun 03 23:38:08.436103 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.436086    2577 scope.go:117] "RemoveContainer" containerID="ad73e2893503273a81b9e8378846b9e9c0e97aa12ccdc3efb4959dd8eacc0e63"
Jun 03 23:38:08.436482 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.436459    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dz6bj"]
Jun 03 23:38:08.436565 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:38:08.436482    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad73e2893503273a81b9e8378846b9e9c0e97aa12ccdc3efb4959dd8eacc0e63\": container with ID starting with ad73e2893503273a81b9e8378846b9e9c0e97aa12ccdc3efb4959dd8eacc0e63 not found: ID does not exist" containerID="ad73e2893503273a81b9e8378846b9e9c0e97aa12ccdc3efb4959dd8eacc0e63"
Jun 03 23:38:08.436565 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.436506    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad73e2893503273a81b9e8378846b9e9c0e97aa12ccdc3efb4959dd8eacc0e63"} err="failed to get container status \"ad73e2893503273a81b9e8378846b9e9c0e97aa12ccdc3efb4959dd8eacc0e63\": rpc error: code = NotFound desc = could not find container \"ad73e2893503273a81b9e8378846b9e9c0e97aa12ccdc3efb4959dd8eacc0e63\": container with ID starting with ad73e2893503273a81b9e8378846b9e9c0e97aa12ccdc3efb4959dd8eacc0e63 not found: ID does not exist"
Jun 03 23:38:08.439631 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.439614    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dz6bj"]
Jun 03 23:38:08.449133 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.449114    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dlnvb"]
Jun 03 23:38:08.453119 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:08.453099    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dlnvb"]
Jun 03 23:38:10.279905 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:10.279864    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d451586-8619-4745-8b7a-9836032b778e" path="/var/lib/kubelet/pods/0d451586-8619-4745-8b7a-9836032b778e/volumes"
Jun 03 23:38:10.280395 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:10.280277    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74848bec-2464-41b5-8f10-0a3f9f06cf33" path="/var/lib/kubelet/pods/74848bec-2464-41b5-8f10-0a3f9f06cf33/volumes"
Jun 03 23:38:10.933983 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:10.933951    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-769b4b9849-ks8kb"]
Jun 03 23:38:10.934273 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:10.934259    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74848bec-2464-41b5-8f10-0a3f9f06cf33" containerName="authorino"
Jun 03 23:38:10.934329 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:10.934274    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="74848bec-2464-41b5-8f10-0a3f9f06cf33" containerName="authorino"
Jun 03 23:38:10.934329 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:10.934288    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d451586-8619-4745-8b7a-9836032b778e" containerName="authorino"
Jun 03 23:38:10.934329 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:10.934294    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d451586-8619-4745-8b7a-9836032b778e" containerName="authorino"
Jun 03 23:38:10.934427 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:10.934343    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d451586-8619-4745-8b7a-9836032b778e" containerName="authorino"
Jun 03 23:38:10.934427 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:10.934354    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="74848bec-2464-41b5-8f10-0a3f9f06cf33" containerName="authorino"
Jun 03 23:38:10.938881 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:10.938859    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769b4b9849-ks8kb"
Jun 03 23:38:10.941527 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:10.941504    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-z7hml\""
Jun 03 23:38:10.942127 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:10.942113    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jun 03 23:38:10.961595 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:10.961571    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-769b4b9849-ks8kb"]
Jun 03 23:38:11.016879 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:11.016849    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtxm2\" (UniqueName: \"kubernetes.io/projected/b59ad3a7-6f41-47f1-af46-771b820cdea9-kube-api-access-xtxm2\") pod \"maas-controller-769b4b9849-ks8kb\" (UID: \"b59ad3a7-6f41-47f1-af46-771b820cdea9\") " pod="opendatahub/maas-controller-769b4b9849-ks8kb"
Jun 03 23:38:11.117439 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:11.117408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtxm2\" (UniqueName: \"kubernetes.io/projected/b59ad3a7-6f41-47f1-af46-771b820cdea9-kube-api-access-xtxm2\") pod \"maas-controller-769b4b9849-ks8kb\" (UID: \"b59ad3a7-6f41-47f1-af46-771b820cdea9\") " pod="opendatahub/maas-controller-769b4b9849-ks8kb"
Jun 03 23:38:11.123465 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:11.123442    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-85788f9b67-ntcmz"]
Jun 03 23:38:11.126674 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:11.126657    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85788f9b67-ntcmz"
Jun 03 23:38:11.129322 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:11.129279    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtxm2\" (UniqueName: \"kubernetes.io/projected/b59ad3a7-6f41-47f1-af46-771b820cdea9-kube-api-access-xtxm2\") pod \"maas-controller-769b4b9849-ks8kb\" (UID: \"b59ad3a7-6f41-47f1-af46-771b820cdea9\") " pod="opendatahub/maas-controller-769b4b9849-ks8kb"
Jun 03 23:38:11.134997 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:11.134978    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85788f9b67-ntcmz"]
Jun 03 23:38:11.218433 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:11.218365    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gnxn\" (UniqueName: \"kubernetes.io/projected/c6508b87-f417-4802-9c93-cbe67e8a1e53-kube-api-access-8gnxn\") pod \"maas-controller-85788f9b67-ntcmz\" (UID: \"c6508b87-f417-4802-9c93-cbe67e8a1e53\") " pod="opendatahub/maas-controller-85788f9b67-ntcmz"
Jun 03 23:38:11.249311 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:11.249288    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769b4b9849-ks8kb"
Jun 03 23:38:11.319379 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:11.319348    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gnxn\" (UniqueName: \"kubernetes.io/projected/c6508b87-f417-4802-9c93-cbe67e8a1e53-kube-api-access-8gnxn\") pod \"maas-controller-85788f9b67-ntcmz\" (UID: \"c6508b87-f417-4802-9c93-cbe67e8a1e53\") " pod="opendatahub/maas-controller-85788f9b67-ntcmz"
Jun 03 23:38:11.327696 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:11.327668    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gnxn\" (UniqueName: \"kubernetes.io/projected/c6508b87-f417-4802-9c93-cbe67e8a1e53-kube-api-access-8gnxn\") pod \"maas-controller-85788f9b67-ntcmz\" (UID: \"c6508b87-f417-4802-9c93-cbe67e8a1e53\") " pod="opendatahub/maas-controller-85788f9b67-ntcmz"
Jun 03 23:38:11.444359 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:11.444329    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85788f9b67-ntcmz"
Jun 03 23:38:11.571576 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:11.571537    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85788f9b67-ntcmz"]
Jun 03 23:38:11.574362 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:38:11.574337    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6508b87_f417_4802_9c93_cbe67e8a1e53.slice/crio-0d0edca2498f33894303020602eef7af9fbdf09ba1fab3f0e832be2788577d49 WatchSource:0}: Error finding container 0d0edca2498f33894303020602eef7af9fbdf09ba1fab3f0e832be2788577d49: Status 404 returned error can't find the container with id 0d0edca2498f33894303020602eef7af9fbdf09ba1fab3f0e832be2788577d49
Jun 03 23:38:11.582833 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:11.582795    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-769b4b9849-ks8kb"]
Jun 03 23:38:11.585150 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:38:11.585122    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb59ad3a7_6f41_47f1_af46_771b820cdea9.slice/crio-0f46ee6d5df137a755ed50b89375bb051f996f4c520283d43bf185daf9dea9b0 WatchSource:0}: Error finding container 0f46ee6d5df137a755ed50b89375bb051f996f4c520283d43bf185daf9dea9b0: Status 404 returned error can't find the container with id 0f46ee6d5df137a755ed50b89375bb051f996f4c520283d43bf185daf9dea9b0
Jun 03 23:38:12.441489 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:12.441397    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85788f9b67-ntcmz" event={"ID":"c6508b87-f417-4802-9c93-cbe67e8a1e53","Type":"ContainerStarted","Data":"0d0edca2498f33894303020602eef7af9fbdf09ba1fab3f0e832be2788577d49"}
Jun 03 23:38:12.444301 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:12.444224    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769b4b9849-ks8kb" event={"ID":"b59ad3a7-6f41-47f1-af46-771b820cdea9","Type":"ContainerStarted","Data":"0f46ee6d5df137a755ed50b89375bb051f996f4c520283d43bf185daf9dea9b0"}
Jun 03 23:38:15.459247 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:15.459183    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769b4b9849-ks8kb" event={"ID":"b59ad3a7-6f41-47f1-af46-771b820cdea9","Type":"ContainerStarted","Data":"b136bc48a6b56b8ba0a07a72c7cf70e979d8b046db3b161d0af64ea8bbd38948"}
Jun 03 23:38:15.459770 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:15.459330    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-769b4b9849-ks8kb"
Jun 03 23:38:15.460830 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:15.460802    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85788f9b67-ntcmz" event={"ID":"c6508b87-f417-4802-9c93-cbe67e8a1e53","Type":"ContainerStarted","Data":"3493a162b57db02482dc859133db6087cd0b2abbfd76f775de6259a37ea02b32"}
Jun 03 23:38:15.460967 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:15.460951    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-85788f9b67-ntcmz"
Jun 03 23:38:15.478553 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:15.478505    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-769b4b9849-ks8kb" podStartSLOduration=2.422289573 podStartE2EDuration="5.478488687s" totalImagesPullingTime="3.056199114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:38:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:38:15.477479539 +0000 UTC m=+817.640157612" watchObservedRunningTime="2026-06-03 23:38:15.478488687 +0000 UTC m=+817.641166742"
Jun 03 23:38:15.498454 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:15.498404    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-85788f9b67-ntcmz" podStartSLOduration=1.423698071 podStartE2EDuration="4.498386349s" totalImagesPullingTime="3.074688278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:38:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:38:15.497973933 +0000 UTC m=+817.660652008" watchObservedRunningTime="2026-06-03 23:38:15.498386349 +0000 UTC m=+817.661064424"
Jun 03 23:38:26.471631 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:26.471600    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-769b4b9849-ks8kb"
Jun 03 23:38:26.472046 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:26.471667    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-85788f9b67-ntcmz"
Jun 03 23:38:26.530494 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:26.530454    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-769b4b9849-ks8kb"]
Jun 03 23:38:26.530655 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:26.530633    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-769b4b9849-ks8kb" podUID="b59ad3a7-6f41-47f1-af46-771b820cdea9" containerName="manager" containerID="cri-o://b136bc48a6b56b8ba0a07a72c7cf70e979d8b046db3b161d0af64ea8bbd38948" gracePeriod=30
Jun 03 23:38:26.771463 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:26.771442    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769b4b9849-ks8kb"
Jun 03 23:38:26.845149 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:26.845124    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtxm2\" (UniqueName: \"kubernetes.io/projected/b59ad3a7-6f41-47f1-af46-771b820cdea9-kube-api-access-xtxm2\") pod \"b59ad3a7-6f41-47f1-af46-771b820cdea9\" (UID: \"b59ad3a7-6f41-47f1-af46-771b820cdea9\") "
Jun 03 23:38:26.847198 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:26.847176    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b59ad3a7-6f41-47f1-af46-771b820cdea9-kube-api-access-xtxm2" (OuterVolumeSpecName: "kube-api-access-xtxm2") pod "b59ad3a7-6f41-47f1-af46-771b820cdea9" (UID: "b59ad3a7-6f41-47f1-af46-771b820cdea9"). InnerVolumeSpecName "kube-api-access-xtxm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 23:38:26.946131 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:26.946108    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtxm2\" (UniqueName: \"kubernetes.io/projected/b59ad3a7-6f41-47f1-af46-771b820cdea9-kube-api-access-xtxm2\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:38:27.513143 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:27.513099    2577 generic.go:356] "Generic (PLEG): container finished" podID="b59ad3a7-6f41-47f1-af46-771b820cdea9" containerID="b136bc48a6b56b8ba0a07a72c7cf70e979d8b046db3b161d0af64ea8bbd38948" exitCode=0
Jun 03 23:38:27.513551 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:27.513157    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769b4b9849-ks8kb"
Jun 03 23:38:27.513551 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:27.513155    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769b4b9849-ks8kb" event={"ID":"b59ad3a7-6f41-47f1-af46-771b820cdea9","Type":"ContainerDied","Data":"b136bc48a6b56b8ba0a07a72c7cf70e979d8b046db3b161d0af64ea8bbd38948"}
Jun 03 23:38:27.513551 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:27.513273    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769b4b9849-ks8kb" event={"ID":"b59ad3a7-6f41-47f1-af46-771b820cdea9","Type":"ContainerDied","Data":"0f46ee6d5df137a755ed50b89375bb051f996f4c520283d43bf185daf9dea9b0"}
Jun 03 23:38:27.513551 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:27.513303    2577 scope.go:117] "RemoveContainer" containerID="b136bc48a6b56b8ba0a07a72c7cf70e979d8b046db3b161d0af64ea8bbd38948"
Jun 03 23:38:27.521386 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:27.521366    2577 scope.go:117] "RemoveContainer" containerID="b136bc48a6b56b8ba0a07a72c7cf70e979d8b046db3b161d0af64ea8bbd38948"
Jun 03 23:38:27.521645 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:38:27.521625    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b136bc48a6b56b8ba0a07a72c7cf70e979d8b046db3b161d0af64ea8bbd38948\": container with ID starting with b136bc48a6b56b8ba0a07a72c7cf70e979d8b046db3b161d0af64ea8bbd38948 not found: ID does not exist" containerID="b136bc48a6b56b8ba0a07a72c7cf70e979d8b046db3b161d0af64ea8bbd38948"
Jun 03 23:38:27.521707 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:27.521652    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b136bc48a6b56b8ba0a07a72c7cf70e979d8b046db3b161d0af64ea8bbd38948"} err="failed to get container status \"b136bc48a6b56b8ba0a07a72c7cf70e979d8b046db3b161d0af64ea8bbd38948\": rpc error: code = NotFound desc = could not find container \"b136bc48a6b56b8ba0a07a72c7cf70e979d8b046db3b161d0af64ea8bbd38948\": container with ID starting with b136bc48a6b56b8ba0a07a72c7cf70e979d8b046db3b161d0af64ea8bbd38948 not found: ID does not exist"
Jun 03 23:38:27.533472 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:27.533451    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-769b4b9849-ks8kb"]
Jun 03 23:38:27.537186 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:27.537165    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-769b4b9849-ks8kb"]
Jun 03 23:38:28.278504 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:28.278465    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b59ad3a7-6f41-47f1-af46-771b820cdea9" path="/var/lib/kubelet/pods/b59ad3a7-6f41-47f1-af46-771b820cdea9/volumes"
Jun 03 23:38:45.992709 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:45.992672    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-85945fb4bf-qwmr9"]
Jun 03 23:38:45.993151 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:45.993020    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b59ad3a7-6f41-47f1-af46-771b820cdea9" containerName="manager"
Jun 03 23:38:45.993151 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:45.993033    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b59ad3a7-6f41-47f1-af46-771b820cdea9" containerName="manager"
Jun 03 23:38:45.993151 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:45.993095    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b59ad3a7-6f41-47f1-af46-771b820cdea9" containerName="manager"
Jun 03 23:38:46.037370 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.037340    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-85945fb4bf-qwmr9"]
Jun 03 23:38:46.037511 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.037428    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-85945fb4bf-qwmr9"
Jun 03 23:38:46.040013 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.039986    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
Jun 03 23:38:46.040251 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.040235    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-m7h47\""
Jun 03 23:38:46.202874 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.202848    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a73e73c-7cc2-4cef-979c-4d214eb7f745-maas-api-tls\") pod \"maas-api-85945fb4bf-qwmr9\" (UID: \"3a73e73c-7cc2-4cef-979c-4d214eb7f745\") " pod="opendatahub/maas-api-85945fb4bf-qwmr9"
Jun 03 23:38:46.203048 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.202888    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlpwt\" (UniqueName: \"kubernetes.io/projected/3a73e73c-7cc2-4cef-979c-4d214eb7f745-kube-api-access-nlpwt\") pod \"maas-api-85945fb4bf-qwmr9\" (UID: \"3a73e73c-7cc2-4cef-979c-4d214eb7f745\") " pod="opendatahub/maas-api-85945fb4bf-qwmr9"
Jun 03 23:38:46.304381 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.304297    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlpwt\" (UniqueName: \"kubernetes.io/projected/3a73e73c-7cc2-4cef-979c-4d214eb7f745-kube-api-access-nlpwt\") pod \"maas-api-85945fb4bf-qwmr9\" (UID: \"3a73e73c-7cc2-4cef-979c-4d214eb7f745\") " pod="opendatahub/maas-api-85945fb4bf-qwmr9"
Jun 03 23:38:46.304530 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.304432    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a73e73c-7cc2-4cef-979c-4d214eb7f745-maas-api-tls\") pod \"maas-api-85945fb4bf-qwmr9\" (UID: \"3a73e73c-7cc2-4cef-979c-4d214eb7f745\") " pod="opendatahub/maas-api-85945fb4bf-qwmr9"
Jun 03 23:38:46.306905 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.306878    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a73e73c-7cc2-4cef-979c-4d214eb7f745-maas-api-tls\") pod \"maas-api-85945fb4bf-qwmr9\" (UID: \"3a73e73c-7cc2-4cef-979c-4d214eb7f745\") " pod="opendatahub/maas-api-85945fb4bf-qwmr9"
Jun 03 23:38:46.315818 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.315789    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlpwt\" (UniqueName: \"kubernetes.io/projected/3a73e73c-7cc2-4cef-979c-4d214eb7f745-kube-api-access-nlpwt\") pod \"maas-api-85945fb4bf-qwmr9\" (UID: \"3a73e73c-7cc2-4cef-979c-4d214eb7f745\") " pod="opendatahub/maas-api-85945fb4bf-qwmr9"
Jun 03 23:38:46.347998 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.347973    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-85945fb4bf-qwmr9"
Jun 03 23:38:46.486872 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.486726    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-l4xgg"]
Jun 03 23:38:46.490797 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:38:46.490765    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a73e73c_7cc2_4cef_979c_4d214eb7f745.slice/crio-b70f419fde05ac7e855a1119f9bc2259ef3fbba206c5e8fc987fa4717bcdd71a WatchSource:0}: Error finding container b70f419fde05ac7e855a1119f9bc2259ef3fbba206c5e8fc987fa4717bcdd71a: Status 404 returned error can't find the container with id b70f419fde05ac7e855a1119f9bc2259ef3fbba206c5e8fc987fa4717bcdd71a
Jun 03 23:38:46.491251 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.491227    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-85945fb4bf-qwmr9"]
Jun 03 23:38:46.491351 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.491297    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-l4xgg"
Jun 03 23:38:46.493567 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.493548    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jun 03 23:38:46.498793 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.498772    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-l4xgg"]
Jun 03 23:38:46.511236 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.511190    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-sw5zw"]
Jun 03 23:38:46.513227 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.513198    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-sw5zw"
Jun 03 23:38:46.515765 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.515726    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-c2g6z\""
Jun 03 23:38:46.522757 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.522736    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-sw5zw"]
Jun 03 23:38:46.595552 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.595526    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-85945fb4bf-qwmr9" event={"ID":"3a73e73c-7cc2-4cef-979c-4d214eb7f745","Type":"ContainerStarted","Data":"b70f419fde05ac7e855a1119f9bc2259ef3fbba206c5e8fc987fa4717bcdd71a"}
Jun 03 23:38:46.606034 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.605973    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fn54\" (UniqueName: \"kubernetes.io/projected/5e0dcea1-2a68-4857-aa48-6d21a574b4ca-kube-api-access-4fn54\") pod \"payload-processing-7c6c859784-sw5zw\" (UID: \"5e0dcea1-2a68-4857-aa48-6d21a574b4ca\") " pod="openshift-ingress/payload-processing-7c6c859784-sw5zw"
Jun 03 23:38:46.606034 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.606023    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c6zw\" (UniqueName: \"kubernetes.io/projected/11710a07-8ac0-478a-962b-11a2ae70d788-kube-api-access-7c6zw\") pod \"payload-pre-processing-67d7689b65-l4xgg\" (UID: \"11710a07-8ac0-478a-962b-11a2ae70d788\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-l4xgg"
Jun 03 23:38:46.706723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.706689    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fn54\" (UniqueName: \"kubernetes.io/projected/5e0dcea1-2a68-4857-aa48-6d21a574b4ca-kube-api-access-4fn54\") pod \"payload-processing-7c6c859784-sw5zw\" (UID: \"5e0dcea1-2a68-4857-aa48-6d21a574b4ca\") " pod="openshift-ingress/payload-processing-7c6c859784-sw5zw"
Jun 03 23:38:46.706723 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.706726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c6zw\" (UniqueName: \"kubernetes.io/projected/11710a07-8ac0-478a-962b-11a2ae70d788-kube-api-access-7c6zw\") pod \"payload-pre-processing-67d7689b65-l4xgg\" (UID: \"11710a07-8ac0-478a-962b-11a2ae70d788\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-l4xgg"
Jun 03 23:38:46.714362 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.714334    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fn54\" (UniqueName: \"kubernetes.io/projected/5e0dcea1-2a68-4857-aa48-6d21a574b4ca-kube-api-access-4fn54\") pod \"payload-processing-7c6c859784-sw5zw\" (UID: \"5e0dcea1-2a68-4857-aa48-6d21a574b4ca\") " pod="openshift-ingress/payload-processing-7c6c859784-sw5zw"
Jun 03 23:38:46.714463 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.714382    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c6zw\" (UniqueName: \"kubernetes.io/projected/11710a07-8ac0-478a-962b-11a2ae70d788-kube-api-access-7c6zw\") pod \"payload-pre-processing-67d7689b65-l4xgg\" (UID: \"11710a07-8ac0-478a-962b-11a2ae70d788\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-l4xgg"
Jun 03 23:38:46.808259 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.808229    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-l4xgg"
Jun 03 23:38:46.823358 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.823338    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-sw5zw"
Jun 03 23:38:46.949789 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.949755    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-l4xgg"]
Jun 03 23:38:46.953536 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:38:46.953499    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11710a07_8ac0_478a_962b_11a2ae70d788.slice/crio-fea316ff51aef5db2027cb47a54b5267c19942b3e5259a1c925d5c7a6a69ccd5 WatchSource:0}: Error finding container fea316ff51aef5db2027cb47a54b5267c19942b3e5259a1c925d5c7a6a69ccd5: Status 404 returned error can't find the container with id fea316ff51aef5db2027cb47a54b5267c19942b3e5259a1c925d5c7a6a69ccd5
Jun 03 23:38:46.971025 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:46.970992    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-sw5zw"]
Jun 03 23:38:46.973068 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:38:46.973044    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0dcea1_2a68_4857_aa48_6d21a574b4ca.slice/crio-8f7427eaf170d9e866d2f4f2057387ca2e55cf9cb016a30dbfe4558c59f00efa WatchSource:0}: Error finding container 8f7427eaf170d9e866d2f4f2057387ca2e55cf9cb016a30dbfe4558c59f00efa: Status 404 returned error can't find the container with id 8f7427eaf170d9e866d2f4f2057387ca2e55cf9cb016a30dbfe4558c59f00efa
Jun 03 23:38:47.609115 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:47.609074    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-sw5zw" event={"ID":"5e0dcea1-2a68-4857-aa48-6d21a574b4ca","Type":"ContainerStarted","Data":"8f7427eaf170d9e866d2f4f2057387ca2e55cf9cb016a30dbfe4558c59f00efa"}
Jun 03 23:38:47.611076 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:47.611045    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-l4xgg" event={"ID":"11710a07-8ac0-478a-962b-11a2ae70d788","Type":"ContainerStarted","Data":"fea316ff51aef5db2027cb47a54b5267c19942b3e5259a1c925d5c7a6a69ccd5"}
Jun 03 23:38:48.617950 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:48.617908    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-85945fb4bf-qwmr9" event={"ID":"3a73e73c-7cc2-4cef-979c-4d214eb7f745","Type":"ContainerStarted","Data":"885740998db07dc2811cbd7c1e48f99a4c40adfb5684ed06e0d6ce0be6712d83"}
Jun 03 23:38:48.618425 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:48.618050    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-85945fb4bf-qwmr9"
Jun 03 23:38:48.636468 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:48.636425    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-85945fb4bf-qwmr9" podStartSLOduration=2.01325227 podStartE2EDuration="3.636412312s" totalImagesPullingTime="1.623160042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:38:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:38:48.63427769 +0000 UTC m=+850.796955764" watchObservedRunningTime="2026-06-03 23:38:48.636412312 +0000 UTC m=+850.799090386"
Jun 03 23:38:49.624119 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:49.624015    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-l4xgg" event={"ID":"11710a07-8ac0-478a-962b-11a2ae70d788","Type":"ContainerStarted","Data":"5b1500232b5374e3418e2943b167ff7c43db7e65c834e775ee91609b31ed21d2"}
Jun 03 23:38:49.624554 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:49.624124    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-l4xgg"
Jun 03 23:38:49.625494 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:49.625471    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-sw5zw" event={"ID":"5e0dcea1-2a68-4857-aa48-6d21a574b4ca","Type":"ContainerStarted","Data":"c1d70ec18603195477197cb159e481ea6c816b0b91c7284ccb027ad28d993cc7"}
Jun 03 23:38:49.625607 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:49.625590    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-sw5zw"
Jun 03 23:38:49.642027 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:49.641985    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-67d7689b65-l4xgg" podStartSLOduration=1.252650807 podStartE2EDuration="3.641972414s" totalImagesPullingTime="2.389321607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:38:49.639587411 +0000 UTC m=+851.802265489" watchObservedRunningTime="2026-06-03 23:38:49.641972414 +0000 UTC m=+851.804650493"
Jun 03 23:38:49.657260 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:49.657197    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7c6c859784-sw5zw" podStartSLOduration=1.29147971 podStartE2EDuration="3.657186035s" totalImagesPullingTime="2.365706325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:38:49.655301906 +0000 UTC m=+851.817979979" watchObservedRunningTime="2026-06-03 23:38:49.657186035 +0000 UTC m=+851.819864120"
Jun 03 23:38:54.631160 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:54.631128    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-85945fb4bf-qwmr9"
Jun 03 23:38:56.454473 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:56.454435    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 23:38:56.454922 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:56.454630    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak" containerID="cri-o://66218abc23227e91b1e8dbd71bb6f62c19fcaaa4eec41c06c087f4a98c451ba3" gracePeriod=30
Jun 03 23:38:58.091217 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.091190    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 03 23:38:58.213926 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.213851    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rxt5\" (UniqueName: \"kubernetes.io/projected/96b06857-3261-429d-937e-0c8cc45457bb-kube-api-access-2rxt5\") pod \"96b06857-3261-429d-937e-0c8cc45457bb\" (UID: \"96b06857-3261-429d-937e-0c8cc45457bb\") "
Jun 03 23:38:58.215850 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.215829    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96b06857-3261-429d-937e-0c8cc45457bb-kube-api-access-2rxt5" (OuterVolumeSpecName: "kube-api-access-2rxt5") pod "96b06857-3261-429d-937e-0c8cc45457bb" (UID: "96b06857-3261-429d-937e-0c8cc45457bb"). InnerVolumeSpecName "kube-api-access-2rxt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 23:38:58.314907 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.314881    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rxt5\" (UniqueName: \"kubernetes.io/projected/96b06857-3261-429d-937e-0c8cc45457bb-kube-api-access-2rxt5\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:38:58.667373 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.667335    2577 generic.go:356] "Generic (PLEG): container finished" podID="96b06857-3261-429d-937e-0c8cc45457bb" containerID="66218abc23227e91b1e8dbd71bb6f62c19fcaaa4eec41c06c087f4a98c451ba3" exitCode=143
Jun 03 23:38:58.667535 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.667377    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 03 23:38:58.667535 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.667405    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"96b06857-3261-429d-937e-0c8cc45457bb","Type":"ContainerDied","Data":"66218abc23227e91b1e8dbd71bb6f62c19fcaaa4eec41c06c087f4a98c451ba3"}
Jun 03 23:38:58.667535 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.667449    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"96b06857-3261-429d-937e-0c8cc45457bb","Type":"ContainerDied","Data":"38d38544e3f9e99590032f6e73bd00e4cb5e86f1f76633630a4fe05db158d0f6"}
Jun 03 23:38:58.667535 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.667471    2577 scope.go:117] "RemoveContainer" containerID="66218abc23227e91b1e8dbd71bb6f62c19fcaaa4eec41c06c087f4a98c451ba3"
Jun 03 23:38:58.676319 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.676299    2577 scope.go:117] "RemoveContainer" containerID="66218abc23227e91b1e8dbd71bb6f62c19fcaaa4eec41c06c087f4a98c451ba3"
Jun 03 23:38:58.676564 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:38:58.676546    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66218abc23227e91b1e8dbd71bb6f62c19fcaaa4eec41c06c087f4a98c451ba3\": container with ID starting with 66218abc23227e91b1e8dbd71bb6f62c19fcaaa4eec41c06c087f4a98c451ba3 not found: ID does not exist" containerID="66218abc23227e91b1e8dbd71bb6f62c19fcaaa4eec41c06c087f4a98c451ba3"
Jun 03 23:38:58.676616 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.676574    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66218abc23227e91b1e8dbd71bb6f62c19fcaaa4eec41c06c087f4a98c451ba3"} err="failed to get container status \"66218abc23227e91b1e8dbd71bb6f62c19fcaaa4eec41c06c087f4a98c451ba3\": rpc error: code = NotFound desc = could not find container \"66218abc23227e91b1e8dbd71bb6f62c19fcaaa4eec41c06c087f4a98c451ba3\": container with ID starting with 66218abc23227e91b1e8dbd71bb6f62c19fcaaa4eec41c06c087f4a98c451ba3 not found: ID does not exist"
Jun 03 23:38:58.686252 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.685269    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 23:38:58.690136 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.690105    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 23:38:58.715801 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.715778    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 23:38:58.716105 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.716093    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak"
Jun 03 23:38:58.716157 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.716107    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak"
Jun 03 23:38:58.716195 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.716163    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="96b06857-3261-429d-937e-0c8cc45457bb" containerName="keycloak"
Jun 03 23:38:58.720257 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.720240    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 03 23:38:58.724520 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.724499    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\""
Jun 03 23:38:58.724665 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.724647    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\""
Jun 03 23:38:58.724742 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.724722    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\""
Jun 03 23:38:58.725146 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.725130    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-v9dhl\""
Jun 03 23:38:58.725600 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.725584    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\""
Jun 03 23:38:58.737827 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.737803    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 23:38:58.818378 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.818355    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mrh8\" (UniqueName: \"kubernetes.io/projected/0a5555cc-f69f-44fd-a2af-6fb28f42d29e-kube-api-access-6mrh8\") pod \"maas-keycloak-0\" (UID: \"0a5555cc-f69f-44fd-a2af-6fb28f42d29e\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 23:38:58.818496 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.818429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/0a5555cc-f69f-44fd-a2af-6fb28f42d29e-test-realms\") pod \"maas-keycloak-0\" (UID: \"0a5555cc-f69f-44fd-a2af-6fb28f42d29e\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 23:38:58.919533 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.919464    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/0a5555cc-f69f-44fd-a2af-6fb28f42d29e-test-realms\") pod \"maas-keycloak-0\" (UID: \"0a5555cc-f69f-44fd-a2af-6fb28f42d29e\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 23:38:58.919533 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.919501    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mrh8\" (UniqueName: \"kubernetes.io/projected/0a5555cc-f69f-44fd-a2af-6fb28f42d29e-kube-api-access-6mrh8\") pod \"maas-keycloak-0\" (UID: \"0a5555cc-f69f-44fd-a2af-6fb28f42d29e\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 23:38:58.920118 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.920098    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/0a5555cc-f69f-44fd-a2af-6fb28f42d29e-test-realms\") pod \"maas-keycloak-0\" (UID: \"0a5555cc-f69f-44fd-a2af-6fb28f42d29e\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 23:38:58.927421 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:58.927398    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mrh8\" (UniqueName: \"kubernetes.io/projected/0a5555cc-f69f-44fd-a2af-6fb28f42d29e-kube-api-access-6mrh8\") pod \"maas-keycloak-0\" (UID: \"0a5555cc-f69f-44fd-a2af-6fb28f42d29e\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 23:38:59.029758 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:59.029726    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 03 23:38:59.157639 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:59.157616    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 23:38:59.159897 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:38:59.159870    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a5555cc_f69f_44fd_a2af_6fb28f42d29e.slice/crio-a33cc7d22d09d376076a6f9a63138dacb1ebf091c4872bea757e6b3b7632665f WatchSource:0}: Error finding container a33cc7d22d09d376076a6f9a63138dacb1ebf091c4872bea757e6b3b7632665f: Status 404 returned error can't find the container with id a33cc7d22d09d376076a6f9a63138dacb1ebf091c4872bea757e6b3b7632665f
Jun 03 23:38:59.675480 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:59.675446    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"0a5555cc-f69f-44fd-a2af-6fb28f42d29e","Type":"ContainerStarted","Data":"90c1d5433688a1ea0aed2fb19bbf74d5970bf8cbd8840cd96c59a9434beab1e2"}
Jun 03 23:38:59.675617 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:59.675498    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"0a5555cc-f69f-44fd-a2af-6fb28f42d29e","Type":"ContainerStarted","Data":"a33cc7d22d09d376076a6f9a63138dacb1ebf091c4872bea757e6b3b7632665f"}
Jun 03 23:38:59.699390 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:38:59.699333    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.38664802 podStartE2EDuration="1.699313581s" totalImagesPullingTime="312.665561ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:38:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:38:59.698789267 +0000 UTC m=+861.861467355" watchObservedRunningTime="2026-06-03 23:38:59.699313581 +0000 UTC m=+861.861991655"
Jun 03 23:39:00.030162 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:00.030128    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0"
Jun 03 23:39:00.031944 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:00.031905    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0a5555cc-f69f-44fd-a2af-6fb28f42d29e" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused"
Jun 03 23:39:00.279659 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:00.279622    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96b06857-3261-429d-937e-0c8cc45457bb" path="/var/lib/kubelet/pods/96b06857-3261-429d-937e-0c8cc45457bb/volumes"
Jun 03 23:39:00.632760 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:00.632729    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-l4xgg"
Jun 03 23:39:00.632961 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:00.632802    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-sw5zw"
Jun 03 23:39:01.031059 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:01.030955    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0a5555cc-f69f-44fd-a2af-6fb28f42d29e" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused"
Jun 03 23:39:02.030962 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:02.030914    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0a5555cc-f69f-44fd-a2af-6fb28f42d29e" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused"
Jun 03 23:39:03.031305 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:03.031250    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0a5555cc-f69f-44fd-a2af-6fb28f42d29e" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused"
Jun 03 23:39:04.030712 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:04.030663    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0a5555cc-f69f-44fd-a2af-6fb28f42d29e" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused"
Jun 03 23:39:05.031052 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:05.030644    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0a5555cc-f69f-44fd-a2af-6fb28f42d29e" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused"
Jun 03 23:39:06.030958 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:06.030914    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0a5555cc-f69f-44fd-a2af-6fb28f42d29e" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused"
Jun 03 23:39:07.030615 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:07.030568    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0a5555cc-f69f-44fd-a2af-6fb28f42d29e" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused"
Jun 03 23:39:08.030680 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:08.030626    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0a5555cc-f69f-44fd-a2af-6fb28f42d29e" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused"
Jun 03 23:39:09.030540 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:09.030492    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0"
Jun 03 23:39:09.030850 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:09.030814    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0a5555cc-f69f-44fd-a2af-6fb28f42d29e" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused"
Jun 03 23:39:10.030673 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:10.030627    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0a5555cc-f69f-44fd-a2af-6fb28f42d29e" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused"
Jun 03 23:39:11.030586 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:11.030531    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0a5555cc-f69f-44fd-a2af-6fb28f42d29e" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused"
Jun 03 23:39:12.031090 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:12.031045    2577 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0a5555cc-f69f-44fd-a2af-6fb28f42d29e" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.41:9000/health/started\": dial tcp 10.134.0.41:9000: connect: connection refused"
Jun 03 23:39:13.161682 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:13.161644    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0"
Jun 03 23:39:13.180239 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:13.180171    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="0a5555cc-f69f-44fd-a2af-6fb28f42d29e" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 03 23:39:23.168657 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:23.168557    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0"
Jun 03 23:39:33.876173 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:33.876140    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-69f4977454-rk5bl"]
Jun 03 23:39:33.879866 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:33.879846    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69f4977454-rk5bl"
Jun 03 23:39:33.882023 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:33.882002    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\""
Jun 03 23:39:33.889073 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:33.889052    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69f4977454-rk5bl"]
Jun 03 23:39:34.036752 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:34.036712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/18693873-4537-4403-ac12-823125ec5cd0-tls-cert\") pod \"authorino-69f4977454-rk5bl\" (UID: \"18693873-4537-4403-ac12-823125ec5cd0\") " pod="kuadrant-system/authorino-69f4977454-rk5bl"
Jun 03 23:39:34.036936 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:34.036764    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/18693873-4537-4403-ac12-823125ec5cd0-oidc-ca\") pod \"authorino-69f4977454-rk5bl\" (UID: \"18693873-4537-4403-ac12-823125ec5cd0\") " pod="kuadrant-system/authorino-69f4977454-rk5bl"
Jun 03 23:39:34.036936 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:34.036858    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbgwd\" (UniqueName: \"kubernetes.io/projected/18693873-4537-4403-ac12-823125ec5cd0-kube-api-access-bbgwd\") pod \"authorino-69f4977454-rk5bl\" (UID: \"18693873-4537-4403-ac12-823125ec5cd0\") " pod="kuadrant-system/authorino-69f4977454-rk5bl"
Jun 03 23:39:34.137758 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:34.137686    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbgwd\" (UniqueName: \"kubernetes.io/projected/18693873-4537-4403-ac12-823125ec5cd0-kube-api-access-bbgwd\") pod \"authorino-69f4977454-rk5bl\" (UID: \"18693873-4537-4403-ac12-823125ec5cd0\") " pod="kuadrant-system/authorino-69f4977454-rk5bl"
Jun 03 23:39:34.137758 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:34.137746    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/18693873-4537-4403-ac12-823125ec5cd0-tls-cert\") pod \"authorino-69f4977454-rk5bl\" (UID: \"18693873-4537-4403-ac12-823125ec5cd0\") " pod="kuadrant-system/authorino-69f4977454-rk5bl"
Jun 03 23:39:34.137913 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:34.137784    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/18693873-4537-4403-ac12-823125ec5cd0-oidc-ca\") pod \"authorino-69f4977454-rk5bl\" (UID: \"18693873-4537-4403-ac12-823125ec5cd0\") " pod="kuadrant-system/authorino-69f4977454-rk5bl"
Jun 03 23:39:34.138450 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:34.138429    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/18693873-4537-4403-ac12-823125ec5cd0-oidc-ca\") pod \"authorino-69f4977454-rk5bl\" (UID: \"18693873-4537-4403-ac12-823125ec5cd0\") " pod="kuadrant-system/authorino-69f4977454-rk5bl"
Jun 03 23:39:34.140237 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:34.140198    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/18693873-4537-4403-ac12-823125ec5cd0-tls-cert\") pod \"authorino-69f4977454-rk5bl\" (UID: \"18693873-4537-4403-ac12-823125ec5cd0\") " pod="kuadrant-system/authorino-69f4977454-rk5bl"
Jun 03 23:39:34.145048 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:34.145027    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbgwd\" (UniqueName: \"kubernetes.io/projected/18693873-4537-4403-ac12-823125ec5cd0-kube-api-access-bbgwd\") pod \"authorino-69f4977454-rk5bl\" (UID: \"18693873-4537-4403-ac12-823125ec5cd0\") " pod="kuadrant-system/authorino-69f4977454-rk5bl"
Jun 03 23:39:34.189784 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:34.189766    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69f4977454-rk5bl"
Jun 03 23:39:34.320688 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:34.320661    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69f4977454-rk5bl"]
Jun 03 23:39:34.323071 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:39:34.323048    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18693873_4537_4403_ac12_823125ec5cd0.slice/crio-aa057bb19d9f659dc0a8ab4b5c53c0d77cb297260ffdf80ecd515d8588f910bf WatchSource:0}: Error finding container aa057bb19d9f659dc0a8ab4b5c53c0d77cb297260ffdf80ecd515d8588f910bf: Status 404 returned error can't find the container with id aa057bb19d9f659dc0a8ab4b5c53c0d77cb297260ffdf80ecd515d8588f910bf
Jun 03 23:39:34.873516 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:34.873475    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69f4977454-rk5bl" event={"ID":"18693873-4537-4403-ac12-823125ec5cd0","Type":"ContainerStarted","Data":"4ab259854552e27cce9f5c9fa44df9918ddb3566b8da08b645365cf491bc63b4"}
Jun 03 23:39:34.873675 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:34.873528    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69f4977454-rk5bl" event={"ID":"18693873-4537-4403-ac12-823125ec5cd0","Type":"ContainerStarted","Data":"aa057bb19d9f659dc0a8ab4b5c53c0d77cb297260ffdf80ecd515d8588f910bf"}
Jun 03 23:39:34.890552 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:34.890501    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-69f4977454-rk5bl" podStartSLOduration=1.497086598 podStartE2EDuration="1.890478694s" totalImagesPullingTime="393.392096ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:39:34.888950776 +0000 UTC m=+897.051628851" watchObservedRunningTime="2026-06-03 23:39:34.890478694 +0000 UTC m=+897.053156779"
Jun 03 23:39:34.915954 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:34.915925    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64fc754b9b-2mx2t"]
Jun 03 23:39:34.916340 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:34.916083    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-64fc754b9b-2mx2t" podUID="984592c6-c108-4a27-aa8a-0b78d2c33bf2" containerName="authorino" containerID="cri-o://0efad4d7f92d1ec1e32f474e814f3a1324b153a81aa1ad9e5a94ffd2f232b6ec" gracePeriod=30
Jun 03 23:39:35.191684 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:35.191653    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64fc754b9b-2mx2t"
Jun 03 23:39:35.348662 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:35.348634    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/984592c6-c108-4a27-aa8a-0b78d2c33bf2-tls-cert\") pod \"984592c6-c108-4a27-aa8a-0b78d2c33bf2\" (UID: \"984592c6-c108-4a27-aa8a-0b78d2c33bf2\") "
Jun 03 23:39:35.348844 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:35.348762    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhzdb\" (UniqueName: \"kubernetes.io/projected/984592c6-c108-4a27-aa8a-0b78d2c33bf2-kube-api-access-xhzdb\") pod \"984592c6-c108-4a27-aa8a-0b78d2c33bf2\" (UID: \"984592c6-c108-4a27-aa8a-0b78d2c33bf2\") "
Jun 03 23:39:35.350666 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:35.350633    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/984592c6-c108-4a27-aa8a-0b78d2c33bf2-kube-api-access-xhzdb" (OuterVolumeSpecName: "kube-api-access-xhzdb") pod "984592c6-c108-4a27-aa8a-0b78d2c33bf2" (UID: "984592c6-c108-4a27-aa8a-0b78d2c33bf2"). InnerVolumeSpecName "kube-api-access-xhzdb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 23:39:35.359078 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:35.359054    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/984592c6-c108-4a27-aa8a-0b78d2c33bf2-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "984592c6-c108-4a27-aa8a-0b78d2c33bf2" (UID: "984592c6-c108-4a27-aa8a-0b78d2c33bf2"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 03 23:39:35.449649 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:35.449620    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhzdb\" (UniqueName: \"kubernetes.io/projected/984592c6-c108-4a27-aa8a-0b78d2c33bf2-kube-api-access-xhzdb\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:39:35.449649 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:35.449646    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/984592c6-c108-4a27-aa8a-0b78d2c33bf2-tls-cert\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:39:35.879134 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:35.879099    2577 generic.go:356] "Generic (PLEG): container finished" podID="984592c6-c108-4a27-aa8a-0b78d2c33bf2" containerID="0efad4d7f92d1ec1e32f474e814f3a1324b153a81aa1ad9e5a94ffd2f232b6ec" exitCode=0
Jun 03 23:39:35.879338 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:35.879142    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64fc754b9b-2mx2t"
Jun 03 23:39:35.879338 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:35.879190    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64fc754b9b-2mx2t" event={"ID":"984592c6-c108-4a27-aa8a-0b78d2c33bf2","Type":"ContainerDied","Data":"0efad4d7f92d1ec1e32f474e814f3a1324b153a81aa1ad9e5a94ffd2f232b6ec"}
Jun 03 23:39:35.879338 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:35.879260    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64fc754b9b-2mx2t" event={"ID":"984592c6-c108-4a27-aa8a-0b78d2c33bf2","Type":"ContainerDied","Data":"35ececf6c75f536eb1c9dd557524ab8d2ae94fad518a535945ac7278e1cf9851"}
Jun 03 23:39:35.879338 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:35.879284    2577 scope.go:117] "RemoveContainer" containerID="0efad4d7f92d1ec1e32f474e814f3a1324b153a81aa1ad9e5a94ffd2f232b6ec"
Jun 03 23:39:35.887788 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:35.887770    2577 scope.go:117] "RemoveContainer" containerID="0efad4d7f92d1ec1e32f474e814f3a1324b153a81aa1ad9e5a94ffd2f232b6ec"
Jun 03 23:39:35.888036 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:39:35.888016    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0efad4d7f92d1ec1e32f474e814f3a1324b153a81aa1ad9e5a94ffd2f232b6ec\": container with ID starting with 0efad4d7f92d1ec1e32f474e814f3a1324b153a81aa1ad9e5a94ffd2f232b6ec not found: ID does not exist" containerID="0efad4d7f92d1ec1e32f474e814f3a1324b153a81aa1ad9e5a94ffd2f232b6ec"
Jun 03 23:39:35.888079 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:35.888043    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0efad4d7f92d1ec1e32f474e814f3a1324b153a81aa1ad9e5a94ffd2f232b6ec"} err="failed to get container status \"0efad4d7f92d1ec1e32f474e814f3a1324b153a81aa1ad9e5a94ffd2f232b6ec\": rpc error: code = NotFound desc = could not find container \"0efad4d7f92d1ec1e32f474e814f3a1324b153a81aa1ad9e5a94ffd2f232b6ec\": container with ID starting with 0efad4d7f92d1ec1e32f474e814f3a1324b153a81aa1ad9e5a94ffd2f232b6ec not found: ID does not exist"
Jun 03 23:39:35.899890 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:35.899866    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64fc754b9b-2mx2t"]
Jun 03 23:39:35.904585 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:35.904564    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-64fc754b9b-2mx2t"]
Jun 03 23:39:36.278838 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:36.278767    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="984592c6-c108-4a27-aa8a-0b78d2c33bf2" path="/var/lib/kubelet/pods/984592c6-c108-4a27-aa8a-0b78d2c33bf2/volumes"
Jun 03 23:39:38.291578 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:38.291551    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 03 23:39:38.291978 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:38.291939    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 03 23:39:39.174302 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.174269    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"]
Jun 03 23:39:39.174630 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.174614    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="984592c6-c108-4a27-aa8a-0b78d2c33bf2" containerName="authorino"
Jun 03 23:39:39.174681 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.174633    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="984592c6-c108-4a27-aa8a-0b78d2c33bf2" containerName="authorino"
Jun 03 23:39:39.174724 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.174691    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="984592c6-c108-4a27-aa8a-0b78d2c33bf2" containerName="authorino"
Jun 03 23:39:39.179018 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.178997    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.181414 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.181391    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jun 03 23:39:39.181516 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.181441    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jun 03 23:39:39.181966 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.181949    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wwhdp\""
Jun 03 23:39:39.182026 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.182003    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jun 03 23:39:39.201442 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.201417    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"]
Jun 03 23:39:39.280005 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.279975    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9fa1ec5a-f899-4c57-83fe-6a15992694b7-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.280122 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.280030    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fa1ec5a-f899-4c57-83fe-6a15992694b7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.280122 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.280103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fa1ec5a-f899-4c57-83fe-6a15992694b7-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.280221 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.280128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fa1ec5a-f899-4c57-83fe-6a15992694b7-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.280221 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.280147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fa1ec5a-f899-4c57-83fe-6a15992694b7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.280308 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.280228    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqqpj\" (UniqueName: \"kubernetes.io/projected/9fa1ec5a-f899-4c57-83fe-6a15992694b7-kube-api-access-nqqpj\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.381158 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.381132    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fa1ec5a-f899-4c57-83fe-6a15992694b7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.381158 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.381158    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fa1ec5a-f899-4c57-83fe-6a15992694b7-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.381645 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.381177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fa1ec5a-f899-4c57-83fe-6a15992694b7-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.381645 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.381195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fa1ec5a-f899-4c57-83fe-6a15992694b7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.381645 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.381249    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqqpj\" (UniqueName: \"kubernetes.io/projected/9fa1ec5a-f899-4c57-83fe-6a15992694b7-kube-api-access-nqqpj\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.381645 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.381301    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9fa1ec5a-f899-4c57-83fe-6a15992694b7-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.381807 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.381644    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fa1ec5a-f899-4c57-83fe-6a15992694b7-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.381807 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.381700    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fa1ec5a-f899-4c57-83fe-6a15992694b7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.381807 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.381728    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9fa1ec5a-f899-4c57-83fe-6a15992694b7-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.383419 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.383398    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fa1ec5a-f899-4c57-83fe-6a15992694b7-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.383774 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.383754    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fa1ec5a-f899-4c57-83fe-6a15992694b7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.393564 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.393542    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqqpj\" (UniqueName: \"kubernetes.io/projected/9fa1ec5a-f899-4c57-83fe-6a15992694b7-kube-api-access-nqqpj\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp\" (UID: \"9fa1ec5a-f899-4c57-83fe-6a15992694b7\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.491051 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.490995    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:39.626825 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.626802    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"]
Jun 03 23:39:39.628582 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:39:39.628550    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fa1ec5a_f899_4c57_83fe_6a15992694b7.slice/crio-ef1ec54e1e3ec41c9d7ec11e3dcbd6d3ef2a25d0a3190d86f0b8f64f0dac43bc WatchSource:0}: Error finding container ef1ec54e1e3ec41c9d7ec11e3dcbd6d3ef2a25d0a3190d86f0b8f64f0dac43bc: Status 404 returned error can't find the container with id ef1ec54e1e3ec41c9d7ec11e3dcbd6d3ef2a25d0a3190d86f0b8f64f0dac43bc
Jun 03 23:39:39.898146 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:39.898106    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp" event={"ID":"9fa1ec5a-f899-4c57-83fe-6a15992694b7","Type":"ContainerStarted","Data":"ef1ec54e1e3ec41c9d7ec11e3dcbd6d3ef2a25d0a3190d86f0b8f64f0dac43bc"}
Jun 03 23:39:42.914169 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:42.914127    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp" event={"ID":"9fa1ec5a-f899-4c57-83fe-6a15992694b7","Type":"ContainerStarted","Data":"9cfe06486e3566ee9a47c2fb7568fc7fa10a27afe503976cbcc4b5dddb122bb5"}
Jun 03 23:39:43.919754 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:43.919719    2577 generic.go:356] "Generic (PLEG): container finished" podID="9fa1ec5a-f899-4c57-83fe-6a15992694b7" containerID="9cfe06486e3566ee9a47c2fb7568fc7fa10a27afe503976cbcc4b5dddb122bb5" exitCode=0
Jun 03 23:39:43.920238 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:43.919808    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp" event={"ID":"9fa1ec5a-f899-4c57-83fe-6a15992694b7","Type":"ContainerDied","Data":"9cfe06486e3566ee9a47c2fb7568fc7fa10a27afe503976cbcc4b5dddb122bb5"}
Jun 03 23:39:45.932605 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:45.932566    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp" event={"ID":"9fa1ec5a-f899-4c57-83fe-6a15992694b7","Type":"ContainerStarted","Data":"7a0035b2780ee0770dd31603af269d925284685cde697f196e17ee3585fe2e89"}
Jun 03 23:39:45.952780 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:45.952733    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp" podStartSLOduration=1.884306059 podStartE2EDuration="6.952720405s" totalImagesPullingTime="3.863001532s" totalInitContainerRuntime="1.205412814s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:39:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:39:45.95063199 +0000 UTC m=+908.113310063" watchObservedRunningTime="2026-06-03 23:39:45.952720405 +0000 UTC m=+908.115398480"
Jun 03 23:39:49.491369 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:49.491335    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:49.491734 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:49.491393    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:49.503825 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:49.503802    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:39:49.960995 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:39:49.960966    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp"
Jun 03 23:40:18.294005 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.293974    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"]
Jun 03 23:40:18.297329 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.297308    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.299591 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.299569    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jun 03 23:40:18.312362 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.312342    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"]
Jun 03 23:40:18.369975 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.369945    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/130014e0-cf10-434f-82d4-f2b5aa29455e-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.370125 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.370003    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/130014e0-cf10-434f-82d4-f2b5aa29455e-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.370125 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.370044    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/130014e0-cf10-434f-82d4-f2b5aa29455e-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.370125 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.370116    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/130014e0-cf10-434f-82d4-f2b5aa29455e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.370299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.370139    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/130014e0-cf10-434f-82d4-f2b5aa29455e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.370299 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.370175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll9wh\" (UniqueName: \"kubernetes.io/projected/130014e0-cf10-434f-82d4-f2b5aa29455e-kube-api-access-ll9wh\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.471522 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.471492    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/130014e0-cf10-434f-82d4-f2b5aa29455e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.471704 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.471529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/130014e0-cf10-434f-82d4-f2b5aa29455e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.471704 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.471564    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll9wh\" (UniqueName: \"kubernetes.io/projected/130014e0-cf10-434f-82d4-f2b5aa29455e-kube-api-access-ll9wh\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.471704 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.471606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/130014e0-cf10-434f-82d4-f2b5aa29455e-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.471704 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.471655    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/130014e0-cf10-434f-82d4-f2b5aa29455e-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.471704 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.471683    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/130014e0-cf10-434f-82d4-f2b5aa29455e-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.471964 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.471940    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/130014e0-cf10-434f-82d4-f2b5aa29455e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.472028 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.471967    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/130014e0-cf10-434f-82d4-f2b5aa29455e-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.472085 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.472026    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/130014e0-cf10-434f-82d4-f2b5aa29455e-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.473870 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.473851    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/130014e0-cf10-434f-82d4-f2b5aa29455e-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.474176 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.474156    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/130014e0-cf10-434f-82d4-f2b5aa29455e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.479808 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.479788    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll9wh\" (UniqueName: \"kubernetes.io/projected/130014e0-cf10-434f-82d4-f2b5aa29455e-kube-api-access-ll9wh\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn\" (UID: \"130014e0-cf10-434f-82d4-f2b5aa29455e\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.607152 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.607122    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:40:18.745040 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:18.745004    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"]
Jun 03 23:40:18.748038 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:40:18.748008    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod130014e0_cf10_434f_82d4_f2b5aa29455e.slice/crio-9dbe8683782a0e86d02d4fa76a4aad1294393664994275a3b9cec772f504e971 WatchSource:0}: Error finding container 9dbe8683782a0e86d02d4fa76a4aad1294393664994275a3b9cec772f504e971: Status 404 returned error can't find the container with id 9dbe8683782a0e86d02d4fa76a4aad1294393664994275a3b9cec772f504e971
Jun 03 23:40:19.078430 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:19.078395    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn" event={"ID":"130014e0-cf10-434f-82d4-f2b5aa29455e","Type":"ContainerStarted","Data":"cdda41e8a688aa822d2c8ffc2f9f24a3a0a2638eb26c0cbeeadeb5753f79962d"}
Jun 03 23:40:19.078598 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:19.078448    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn" event={"ID":"130014e0-cf10-434f-82d4-f2b5aa29455e","Type":"ContainerStarted","Data":"9dbe8683782a0e86d02d4fa76a4aad1294393664994275a3b9cec772f504e971"}
Jun 03 23:40:27.287886 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.287844    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"]
Jun 03 23:40:27.291354 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.291333    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.293979 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.293960    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jun 03 23:40:27.303851 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.303819    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"]
Jun 03 23:40:27.340455 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.340427    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.340581 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.340464    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.340581 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.340485    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlf2x\" (UniqueName: \"kubernetes.io/projected/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-kube-api-access-xlf2x\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.340581 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.340551    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.340726 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.340607    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.340726 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.340651    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.445229 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.441432    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.445229 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.441506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.445229 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.441570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.445229 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.441607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlf2x\" (UniqueName: \"kubernetes.io/projected/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-kube-api-access-xlf2x\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.445229 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.441652    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.445229 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.441711    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.445229 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.444617    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.448056 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.446955    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.448056 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.447222    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.448056 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.447792    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.448895 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.448874    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.455925 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.455903    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlf2x\" (UniqueName: \"kubernetes.io/projected/7c37de7e-75b1-4ee8-8d75-4b71b60e81d3-kube-api-access-xlf2x\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-8f4c9\" (UID: \"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.601450 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.601418    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:27.731032 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:27.730859    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"]
Jun 03 23:40:27.733376 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:40:27.733349    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c37de7e_75b1_4ee8_8d75_4b71b60e81d3.slice/crio-5487d398a66d406130c605cfbe9bc9275c7d91d92872b0a32f4ce6ff7b134c11 WatchSource:0}: Error finding container 5487d398a66d406130c605cfbe9bc9275c7d91d92872b0a32f4ce6ff7b134c11: Status 404 returned error can't find the container with id 5487d398a66d406130c605cfbe9bc9275c7d91d92872b0a32f4ce6ff7b134c11
Jun 03 23:40:28.119882 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:28.119842    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9" event={"ID":"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3","Type":"ContainerStarted","Data":"256e4cd0a1739025b3a591ebaf4b49a4244557cb9c4ae01d9776040364a7bfa5"}
Jun 03 23:40:28.119882 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:28.119886    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9" event={"ID":"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3","Type":"ContainerStarted","Data":"5487d398a66d406130c605cfbe9bc9275c7d91d92872b0a32f4ce6ff7b134c11"}
Jun 03 23:40:29.126312 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:29.126268    2577 generic.go:356] "Generic (PLEG): container finished" podID="7c37de7e-75b1-4ee8-8d75-4b71b60e81d3" containerID="256e4cd0a1739025b3a591ebaf4b49a4244557cb9c4ae01d9776040364a7bfa5" exitCode=0
Jun 03 23:40:29.126780 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:29.126348    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9" event={"ID":"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3","Type":"ContainerDied","Data":"256e4cd0a1739025b3a591ebaf4b49a4244557cb9c4ae01d9776040364a7bfa5"}
Jun 03 23:40:30.133317 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:30.133270    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9" event={"ID":"7c37de7e-75b1-4ee8-8d75-4b71b60e81d3","Type":"ContainerStarted","Data":"ff765a669cc76a0fcf187bb029541baf3a90d8100cc2f4fbf620724b69d5dc75"}
Jun 03 23:40:30.155555 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:30.155505    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9" podStartSLOduration=1.798951947 podStartE2EDuration="3.15549067s" totalImagesPullingTime="201.76895ms" totalInitContainerRuntime="1.154769773s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:40:30.154164469 +0000 UTC m=+952.316842542" watchObservedRunningTime="2026-06-03 23:40:30.15549067 +0000 UTC m=+952.318168745"
Jun 03 23:40:37.602260 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:37.602227    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:37.602649 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:37.602274    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:37.614831 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:37.614808    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:40:38.180275 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:40:38.180243    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-8f4c9"
Jun 03 23:41:03.280609 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:03.280573    2577 generic.go:356] "Generic (PLEG): container finished" podID="130014e0-cf10-434f-82d4-f2b5aa29455e" containerID="cdda41e8a688aa822d2c8ffc2f9f24a3a0a2638eb26c0cbeeadeb5753f79962d" exitCode=0
Jun 03 23:41:03.281038 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:03.280642    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn" event={"ID":"130014e0-cf10-434f-82d4-f2b5aa29455e","Type":"ContainerDied","Data":"cdda41e8a688aa822d2c8ffc2f9f24a3a0a2638eb26c0cbeeadeb5753f79962d"}
Jun 03 23:41:04.285930 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:04.285900    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn" event={"ID":"130014e0-cf10-434f-82d4-f2b5aa29455e","Type":"ContainerStarted","Data":"2acad43a8571d112f964df03fa47e76352c11a67b9c1fb42d8c4b541c7d0d222"}
Jun 03 23:41:04.312900 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:04.312852    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn" podStartSLOduration=2.209796183 podStartE2EDuration="46.31283871s" totalImagesPullingTime="178.10684ms" totalInitContainerRuntime="43.924935687s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:40:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:41:04.311712709 +0000 UTC m=+986.474390783" watchObservedRunningTime="2026-06-03 23:41:04.31283871 +0000 UTC m=+986.475516793"
Jun 03 23:41:08.608261 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:08.608221    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:41:08.608762 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:08.608293    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:41:08.620728 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:08.620705    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:41:09.318549 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:09.318523    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn"
Jun 03 23:41:38.125184 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:38.125148    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5c8965df5f-sqxbx"]
Jun 03 23:41:38.128556 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:38.128534    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c8965df5f-sqxbx"
Jun 03 23:41:38.134628 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:38.134606    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c8965df5f-sqxbx"]
Jun 03 23:41:38.205020 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:38.204996    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b2982bb2-2cab-46e6-a48a-ff5799bd537d-tls-cert\") pod \"authorino-5c8965df5f-sqxbx\" (UID: \"b2982bb2-2cab-46e6-a48a-ff5799bd537d\") " pod="kuadrant-system/authorino-5c8965df5f-sqxbx"
Jun 03 23:41:38.205149 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:38.205028    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/b2982bb2-2cab-46e6-a48a-ff5799bd537d-oidc-ca\") pod \"authorino-5c8965df5f-sqxbx\" (UID: \"b2982bb2-2cab-46e6-a48a-ff5799bd537d\") " pod="kuadrant-system/authorino-5c8965df5f-sqxbx"
Jun 03 23:41:38.205149 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:38.205071    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt6t5\" (UniqueName: \"kubernetes.io/projected/b2982bb2-2cab-46e6-a48a-ff5799bd537d-kube-api-access-gt6t5\") pod \"authorino-5c8965df5f-sqxbx\" (UID: \"b2982bb2-2cab-46e6-a48a-ff5799bd537d\") " pod="kuadrant-system/authorino-5c8965df5f-sqxbx"
Jun 03 23:41:38.306254 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:38.306224    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b2982bb2-2cab-46e6-a48a-ff5799bd537d-tls-cert\") pod \"authorino-5c8965df5f-sqxbx\" (UID: \"b2982bb2-2cab-46e6-a48a-ff5799bd537d\") " pod="kuadrant-system/authorino-5c8965df5f-sqxbx"
Jun 03 23:41:38.306398 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:38.306264    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/b2982bb2-2cab-46e6-a48a-ff5799bd537d-oidc-ca\") pod \"authorino-5c8965df5f-sqxbx\" (UID: \"b2982bb2-2cab-46e6-a48a-ff5799bd537d\") " pod="kuadrant-system/authorino-5c8965df5f-sqxbx"
Jun 03 23:41:38.306398 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:38.306321    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt6t5\" (UniqueName: \"kubernetes.io/projected/b2982bb2-2cab-46e6-a48a-ff5799bd537d-kube-api-access-gt6t5\") pod \"authorino-5c8965df5f-sqxbx\" (UID: \"b2982bb2-2cab-46e6-a48a-ff5799bd537d\") " pod="kuadrant-system/authorino-5c8965df5f-sqxbx"
Jun 03 23:41:38.306969 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:38.306948    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/b2982bb2-2cab-46e6-a48a-ff5799bd537d-oidc-ca\") pod \"authorino-5c8965df5f-sqxbx\" (UID: \"b2982bb2-2cab-46e6-a48a-ff5799bd537d\") " pod="kuadrant-system/authorino-5c8965df5f-sqxbx"
Jun 03 23:41:38.308542 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:38.308518    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b2982bb2-2cab-46e6-a48a-ff5799bd537d-tls-cert\") pod \"authorino-5c8965df5f-sqxbx\" (UID: \"b2982bb2-2cab-46e6-a48a-ff5799bd537d\") " pod="kuadrant-system/authorino-5c8965df5f-sqxbx"
Jun 03 23:41:38.313671 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:38.313653    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt6t5\" (UniqueName: \"kubernetes.io/projected/b2982bb2-2cab-46e6-a48a-ff5799bd537d-kube-api-access-gt6t5\") pod \"authorino-5c8965df5f-sqxbx\" (UID: \"b2982bb2-2cab-46e6-a48a-ff5799bd537d\") " pod="kuadrant-system/authorino-5c8965df5f-sqxbx"
Jun 03 23:41:38.439374 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:38.439305    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c8965df5f-sqxbx"
Jun 03 23:41:38.579326 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:38.579299    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c8965df5f-sqxbx"]
Jun 03 23:41:38.581284 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:41:38.581250    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2982bb2_2cab_46e6_a48a_ff5799bd537d.slice/crio-2bb63a3b08e7a0f347da11c5e86689fccff29dea2fd1d686665e5b4b2f49e5bb WatchSource:0}: Error finding container 2bb63a3b08e7a0f347da11c5e86689fccff29dea2fd1d686665e5b4b2f49e5bb: Status 404 returned error can't find the container with id 2bb63a3b08e7a0f347da11c5e86689fccff29dea2fd1d686665e5b4b2f49e5bb
Jun 03 23:41:39.452039 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:39.451995    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c8965df5f-sqxbx" event={"ID":"b2982bb2-2cab-46e6-a48a-ff5799bd537d","Type":"ContainerStarted","Data":"9678355d203201eaa8a2366f6fde5bc0bf8441e2c00da3a8671ff88d35692688"}
Jun 03 23:41:39.452515 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:39.452051    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c8965df5f-sqxbx" event={"ID":"b2982bb2-2cab-46e6-a48a-ff5799bd537d","Type":"ContainerStarted","Data":"2bb63a3b08e7a0f347da11c5e86689fccff29dea2fd1d686665e5b4b2f49e5bb"}
Jun 03 23:41:39.470657 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:39.470597    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5c8965df5f-sqxbx" podStartSLOduration=1.1199358799999999 podStartE2EDuration="1.470577738s" totalImagesPullingTime="350.641858ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:41:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:41:39.469817374 +0000 UTC m=+1021.632495449" watchObservedRunningTime="2026-06-03 23:41:39.470577738 +0000 UTC m=+1021.633255819"
Jun 03 23:41:39.498558 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:39.498521    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69f4977454-rk5bl"]
Jun 03 23:41:39.498788 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:39.498753    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-69f4977454-rk5bl" podUID="18693873-4537-4403-ac12-823125ec5cd0" containerName="authorino" containerID="cri-o://4ab259854552e27cce9f5c9fa44df9918ddb3566b8da08b645365cf491bc63b4" gracePeriod=30
Jun 03 23:41:39.759658 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:39.759631    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69f4977454-rk5bl"
Jun 03 23:41:39.819880 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:39.819839    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/18693873-4537-4403-ac12-823125ec5cd0-tls-cert\") pod \"18693873-4537-4403-ac12-823125ec5cd0\" (UID: \"18693873-4537-4403-ac12-823125ec5cd0\") "
Jun 03 23:41:39.820036 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:39.819885    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/18693873-4537-4403-ac12-823125ec5cd0-oidc-ca\") pod \"18693873-4537-4403-ac12-823125ec5cd0\" (UID: \"18693873-4537-4403-ac12-823125ec5cd0\") "
Jun 03 23:41:39.820036 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:39.819917    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbgwd\" (UniqueName: \"kubernetes.io/projected/18693873-4537-4403-ac12-823125ec5cd0-kube-api-access-bbgwd\") pod \"18693873-4537-4403-ac12-823125ec5cd0\" (UID: \"18693873-4537-4403-ac12-823125ec5cd0\") "
Jun 03 23:41:39.822481 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:39.822432    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18693873-4537-4403-ac12-823125ec5cd0-kube-api-access-bbgwd" (OuterVolumeSpecName: "kube-api-access-bbgwd") pod "18693873-4537-4403-ac12-823125ec5cd0" (UID: "18693873-4537-4403-ac12-823125ec5cd0"). InnerVolumeSpecName "kube-api-access-bbgwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 23:41:39.827730 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:39.827688    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18693873-4537-4403-ac12-823125ec5cd0-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "18693873-4537-4403-ac12-823125ec5cd0" (UID: "18693873-4537-4403-ac12-823125ec5cd0"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 23:41:39.833466 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:39.833441    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18693873-4537-4403-ac12-823125ec5cd0-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "18693873-4537-4403-ac12-823125ec5cd0" (UID: "18693873-4537-4403-ac12-823125ec5cd0"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 03 23:41:39.920749 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:39.920718    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/18693873-4537-4403-ac12-823125ec5cd0-tls-cert\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:41:39.920749 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:39.920745    2577 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/18693873-4537-4403-ac12-823125ec5cd0-oidc-ca\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:41:39.920917 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:39.920760    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbgwd\" (UniqueName: \"kubernetes.io/projected/18693873-4537-4403-ac12-823125ec5cd0-kube-api-access-bbgwd\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:41:40.458363 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:40.458328    2577 generic.go:356] "Generic (PLEG): container finished" podID="18693873-4537-4403-ac12-823125ec5cd0" containerID="4ab259854552e27cce9f5c9fa44df9918ddb3566b8da08b645365cf491bc63b4" exitCode=0
Jun 03 23:41:40.458776 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:40.458370    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69f4977454-rk5bl"
Jun 03 23:41:40.458776 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:40.458414    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69f4977454-rk5bl" event={"ID":"18693873-4537-4403-ac12-823125ec5cd0","Type":"ContainerDied","Data":"4ab259854552e27cce9f5c9fa44df9918ddb3566b8da08b645365cf491bc63b4"}
Jun 03 23:41:40.458776 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:40.458450    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69f4977454-rk5bl" event={"ID":"18693873-4537-4403-ac12-823125ec5cd0","Type":"ContainerDied","Data":"aa057bb19d9f659dc0a8ab4b5c53c0d77cb297260ffdf80ecd515d8588f910bf"}
Jun 03 23:41:40.458776 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:40.458467    2577 scope.go:117] "RemoveContainer" containerID="4ab259854552e27cce9f5c9fa44df9918ddb3566b8da08b645365cf491bc63b4"
Jun 03 23:41:40.467005 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:40.466985    2577 scope.go:117] "RemoveContainer" containerID="4ab259854552e27cce9f5c9fa44df9918ddb3566b8da08b645365cf491bc63b4"
Jun 03 23:41:40.467264 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:41:40.467245    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ab259854552e27cce9f5c9fa44df9918ddb3566b8da08b645365cf491bc63b4\": container with ID starting with 4ab259854552e27cce9f5c9fa44df9918ddb3566b8da08b645365cf491bc63b4 not found: ID does not exist" containerID="4ab259854552e27cce9f5c9fa44df9918ddb3566b8da08b645365cf491bc63b4"
Jun 03 23:41:40.467323 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:40.467274    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ab259854552e27cce9f5c9fa44df9918ddb3566b8da08b645365cf491bc63b4"} err="failed to get container status \"4ab259854552e27cce9f5c9fa44df9918ddb3566b8da08b645365cf491bc63b4\": rpc error: code = NotFound desc = could not find container \"4ab259854552e27cce9f5c9fa44df9918ddb3566b8da08b645365cf491bc63b4\": container with ID starting with 4ab259854552e27cce9f5c9fa44df9918ddb3566b8da08b645365cf491bc63b4 not found: ID does not exist"
Jun 03 23:41:40.479540 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:40.479480    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69f4977454-rk5bl"]
Jun 03 23:41:40.481187 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:40.481166    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-69f4977454-rk5bl"]
Jun 03 23:41:42.279339 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:41:42.279303    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18693873-4537-4403-ac12-823125ec5cd0" path="/var/lib/kubelet/pods/18693873-4537-4403-ac12-823125ec5cd0/volumes"
Jun 03 23:43:05.107856 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:05.107817    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85788f9b67-ntcmz"]
Jun 03 23:43:05.108412 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:05.108010    2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-85788f9b67-ntcmz" podUID="c6508b87-f417-4802-9c93-cbe67e8a1e53" containerName="manager" containerID="cri-o://3493a162b57db02482dc859133db6087cd0b2abbfd76f775de6259a37ea02b32" gracePeriod=30
Jun 03 23:43:05.358005 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:05.357949    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85788f9b67-ntcmz"
Jun 03 23:43:05.518381 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:05.518353    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gnxn\" (UniqueName: \"kubernetes.io/projected/c6508b87-f417-4802-9c93-cbe67e8a1e53-kube-api-access-8gnxn\") pod \"c6508b87-f417-4802-9c93-cbe67e8a1e53\" (UID: \"c6508b87-f417-4802-9c93-cbe67e8a1e53\") "
Jun 03 23:43:05.520124 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:05.520096    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6508b87-f417-4802-9c93-cbe67e8a1e53-kube-api-access-8gnxn" (OuterVolumeSpecName: "kube-api-access-8gnxn") pod "c6508b87-f417-4802-9c93-cbe67e8a1e53" (UID: "c6508b87-f417-4802-9c93-cbe67e8a1e53"). InnerVolumeSpecName "kube-api-access-8gnxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 23:43:05.619151 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:05.619072    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gnxn\" (UniqueName: \"kubernetes.io/projected/c6508b87-f417-4802-9c93-cbe67e8a1e53-kube-api-access-8gnxn\") on node \"ip-10-0-129-110.ec2.internal\" DevicePath \"\""
Jun 03 23:43:05.862888 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:05.862849    2577 generic.go:356] "Generic (PLEG): container finished" podID="c6508b87-f417-4802-9c93-cbe67e8a1e53" containerID="3493a162b57db02482dc859133db6087cd0b2abbfd76f775de6259a37ea02b32" exitCode=0
Jun 03 23:43:05.863050 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:05.862909    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85788f9b67-ntcmz"
Jun 03 23:43:05.863050 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:05.862927    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85788f9b67-ntcmz" event={"ID":"c6508b87-f417-4802-9c93-cbe67e8a1e53","Type":"ContainerDied","Data":"3493a162b57db02482dc859133db6087cd0b2abbfd76f775de6259a37ea02b32"}
Jun 03 23:43:05.863050 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:05.863004    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85788f9b67-ntcmz" event={"ID":"c6508b87-f417-4802-9c93-cbe67e8a1e53","Type":"ContainerDied","Data":"0d0edca2498f33894303020602eef7af9fbdf09ba1fab3f0e832be2788577d49"}
Jun 03 23:43:05.863050 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:05.863027    2577 scope.go:117] "RemoveContainer" containerID="3493a162b57db02482dc859133db6087cd0b2abbfd76f775de6259a37ea02b32"
Jun 03 23:43:05.872175 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:05.872159    2577 scope.go:117] "RemoveContainer" containerID="3493a162b57db02482dc859133db6087cd0b2abbfd76f775de6259a37ea02b32"
Jun 03 23:43:05.872448 ip-10-0-129-110 kubenswrapper[2577]: E0603 23:43:05.872430    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3493a162b57db02482dc859133db6087cd0b2abbfd76f775de6259a37ea02b32\": container with ID starting with 3493a162b57db02482dc859133db6087cd0b2abbfd76f775de6259a37ea02b32 not found: ID does not exist" containerID="3493a162b57db02482dc859133db6087cd0b2abbfd76f775de6259a37ea02b32"
Jun 03 23:43:05.872496 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:05.872456    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3493a162b57db02482dc859133db6087cd0b2abbfd76f775de6259a37ea02b32"} err="failed to get container status \"3493a162b57db02482dc859133db6087cd0b2abbfd76f775de6259a37ea02b32\": rpc error: code = NotFound desc = could not find container \"3493a162b57db02482dc859133db6087cd0b2abbfd76f775de6259a37ea02b32\": container with ID starting with 3493a162b57db02482dc859133db6087cd0b2abbfd76f775de6259a37ea02b32 not found: ID does not exist"
Jun 03 23:43:05.896000 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:05.895976    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85788f9b67-ntcmz"]
Jun 03 23:43:05.912711 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:05.912693    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-85788f9b67-ntcmz"]
Jun 03 23:43:06.280703 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:06.280625    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6508b87-f417-4802-9c93-cbe67e8a1e53" path="/var/lib/kubelet/pods/c6508b87-f417-4802-9c93-cbe67e8a1e53/volumes"
Jun 03 23:43:06.679313 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:06.679281    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-85788f9b67-5n46c"]
Jun 03 23:43:06.679630 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:06.679617    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18693873-4537-4403-ac12-823125ec5cd0" containerName="authorino"
Jun 03 23:43:06.679671 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:06.679632    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="18693873-4537-4403-ac12-823125ec5cd0" containerName="authorino"
Jun 03 23:43:06.679671 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:06.679650    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6508b87-f417-4802-9c93-cbe67e8a1e53" containerName="manager"
Jun 03 23:43:06.679671 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:06.679655    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6508b87-f417-4802-9c93-cbe67e8a1e53" containerName="manager"
Jun 03 23:43:06.679767 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:06.679721    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6508b87-f417-4802-9c93-cbe67e8a1e53" containerName="manager"
Jun 03 23:43:06.679767 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:06.679733    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="18693873-4537-4403-ac12-823125ec5cd0" containerName="authorino"
Jun 03 23:43:06.684088 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:06.684065    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85788f9b67-5n46c"
Jun 03 23:43:06.686401 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:06.686376    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-z7hml\""
Jun 03 23:43:06.686542 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:06.686383    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jun 03 23:43:06.691161 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:06.691140    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85788f9b67-5n46c"]
Jun 03 23:43:06.828849 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:06.828813    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sjxw\" (UniqueName: \"kubernetes.io/projected/79f8e291-c990-4ac4-86b0-6346da1c442c-kube-api-access-9sjxw\") pod \"maas-controller-85788f9b67-5n46c\" (UID: \"79f8e291-c990-4ac4-86b0-6346da1c442c\") " pod="opendatahub/maas-controller-85788f9b67-5n46c"
Jun 03 23:43:06.929941 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:06.929863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sjxw\" (UniqueName: \"kubernetes.io/projected/79f8e291-c990-4ac4-86b0-6346da1c442c-kube-api-access-9sjxw\") pod \"maas-controller-85788f9b67-5n46c\" (UID: \"79f8e291-c990-4ac4-86b0-6346da1c442c\") " pod="opendatahub/maas-controller-85788f9b67-5n46c"
Jun 03 23:43:06.938532 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:06.938499    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sjxw\" (UniqueName: \"kubernetes.io/projected/79f8e291-c990-4ac4-86b0-6346da1c442c-kube-api-access-9sjxw\") pod \"maas-controller-85788f9b67-5n46c\" (UID: \"79f8e291-c990-4ac4-86b0-6346da1c442c\") " pod="opendatahub/maas-controller-85788f9b67-5n46c"
Jun 03 23:43:06.994608 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:06.994572    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85788f9b67-5n46c"
Jun 03 23:43:07.123468 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:07.123442    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85788f9b67-5n46c"]
Jun 03 23:43:07.125340 ip-10-0-129-110 kubenswrapper[2577]: W0603 23:43:07.125308    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f8e291_c990_4ac4_86b0_6346da1c442c.slice/crio-21064ce3f1c401bf482f94bab84669b0a789268380536914e3cd4a1d805cf705 WatchSource:0}: Error finding container 21064ce3f1c401bf482f94bab84669b0a789268380536914e3cd4a1d805cf705: Status 404 returned error can't find the container with id 21064ce3f1c401bf482f94bab84669b0a789268380536914e3cd4a1d805cf705
Jun 03 23:43:07.126614 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:07.126596    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 03 23:43:07.877992 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:07.877961    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85788f9b67-5n46c" event={"ID":"79f8e291-c990-4ac4-86b0-6346da1c442c","Type":"ContainerStarted","Data":"3a9418cca2928a5522f2f564357af1e5d249dcf7177030b963691f0fd4b2324f"}
Jun 03 23:43:07.878407 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:07.878014    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85788f9b67-5n46c" event={"ID":"79f8e291-c990-4ac4-86b0-6346da1c442c","Type":"ContainerStarted","Data":"21064ce3f1c401bf482f94bab84669b0a789268380536914e3cd4a1d805cf705"}
Jun 03 23:43:07.878407 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:07.878096    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-85788f9b67-5n46c"
Jun 03 23:43:07.897924 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:07.897880    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-85788f9b67-5n46c" podStartSLOduration=1.5239960350000001 podStartE2EDuration="1.897867228s" totalImagesPullingTime="373.871193ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-03 23:43:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-03 23:43:07.895875743 +0000 UTC m=+1110.058553820" watchObservedRunningTime="2026-06-03 23:43:07.897867228 +0000 UTC m=+1110.060545301"
Jun 03 23:43:18.888678 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:43:18.888645    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-85788f9b67-5n46c"
Jun 03 23:44:38.322557 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:44:38.322524    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 03 23:44:38.324200 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:44:38.324178    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 03 23:49:38.347756 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:49:38.347720    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 03 23:49:38.349865 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:49:38.349844    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 03 23:54:38.374369 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:54:38.374337    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 03 23:54:38.377593 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:54:38.377572    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 03 23:59:38.400994 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:59:38.400960    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 03 23:59:38.405340 ip-10-0-129-110 kubenswrapper[2577]: I0603 23:59:38.405246    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 04 00:04:38.432321 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:04:38.432290    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 04 00:04:38.437674 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:04:38.437654    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 04 00:05:08.575672 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:08.575636    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5c8965df5f-sqxbx_b2982bb2-2cab-46e6-a48a-ff5799bd537d/authorino/0.log"
Jun 04 00:05:13.888284 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:13.888249    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-85945fb4bf-qwmr9_3a73e73c-7cc2-4cef-979c-4d214eb7f745/maas-api/0.log"
Jun 04 00:05:14.240398 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:14.240323    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-85788f9b67-5n46c_79f8e291-c990-4ac4-86b0-6346da1c442c/manager/0.log"
Jun 04 00:05:14.724561 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:14.724531    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f87df8b57-lsfmk_966fd4b6-a8b2-4b59-b62b-a723ee509b6d/copy-manifests/0.log"
Jun 04 00:05:14.736685 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:14.736653    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f87df8b57-lsfmk_966fd4b6-a8b2-4b59-b62b-a723ee509b6d/manager/0.log"
Jun 04 00:05:16.311789 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:16.311758    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5c8965df5f-sqxbx_b2982bb2-2cab-46e6-a48a-ff5799bd537d/authorino/0.log"
Jun 04 00:05:17.767624 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:17.767590    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57684f659d-2qfpm_3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2/kube-auth-proxy/0.log"
Jun 04 00:05:18.133783 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:18.133749    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-l4xgg_11710a07-8ac0-478a-962b-11a2ae70d788/payload-pre-processing/0.log"
Jun 04 00:05:18.255541 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:18.255513    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-sw5zw_5e0dcea1-2a68-4857-aa48-6d21a574b4ca/payload-processing/0.log"
Jun 04 00:05:18.376987 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:18.376956    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55b9b48c7c-jw6cm_98372747-9a9e-4a1e-ad2f-d274d1db6889/router/0.log"
Jun 04 00:05:18.732782 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:18.732755    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp_9fa1ec5a-f899-4c57-83fe-6a15992694b7/storage-initializer/0.log"
Jun 04 00:05:18.745046 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:18.745025    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-f7xkp_9fa1ec5a-f899-4c57-83fe-6a15992694b7/main/0.log"
Jun 04 00:05:19.224758 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:19.224728    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn_130014e0-cf10-434f-82d4-f2b5aa29455e/storage-initializer/0.log"
Jun 04 00:05:19.236938 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:19.236914    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-nmwkn_130014e0-cf10-434f-82d4-f2b5aa29455e/main/0.log"
Jun 04 00:05:19.352581 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:19.352552    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-8f4c9_7c37de7e-75b1-4ee8-8d75-4b71b60e81d3/storage-initializer/0.log"
Jun 04 00:05:19.364896 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:19.364877    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-8f4c9_7c37de7e-75b1-4ee8-8d75-4b71b60e81d3/main/0.log"
Jun 04 00:05:26.544059 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:26.544019    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9d2cb_55ceda9f-6200-4a21-a050-d290d631043f/global-pull-secret-syncer/0.log"
Jun 04 00:05:26.704616 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:26.704589    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kx2mq_f0a1234d-160e-4994-834f-b87872953ed1/konnectivity-agent/0.log"
Jun 04 00:05:26.726188 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:26.726164    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-110.ec2.internal_a14572b805eb35459a9a5413dc15e576/haproxy/0.log"
Jun 04 00:05:31.903475 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:31.903443    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5c8965df5f-sqxbx_b2982bb2-2cab-46e6-a48a-ff5799bd537d/authorino/0.log"
Jun 04 00:05:34.083674 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:34.083631    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-57769895dd-bhxbc_6125ef6f-2e47-4f8c-9668-274bdffe1ca2/cluster-monitoring-operator/0.log"
Jun 04 00:05:34.236157 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:34.236131    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-9b9fc658-m49nl_8a6149e0-57e9-4741-99df-0198dcaafa03/monitoring-plugin/0.log"
Jun 04 00:05:34.346338 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:34.346272    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7zqrp_8fb4d18f-a068-4cec-8181-bc88350b50c8/node-exporter/0.log"
Jun 04 00:05:34.371929 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:34.367141    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7zqrp_8fb4d18f-a068-4cec-8181-bc88350b50c8/kube-rbac-proxy/0.log"
Jun 04 00:05:34.387703 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:34.387680    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7zqrp_8fb4d18f-a068-4cec-8181-bc88350b50c8/init-textfile/0.log"
Jun 04 00:05:34.869385 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:34.869354    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-vfmd8_800518f3-21a0-4933-9699-be4f6b7dfeb9/prometheus-operator/0.log"
Jun 04 00:05:34.893503 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:34.893481    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-vfmd8_800518f3-21a0-4933-9699-be4f6b7dfeb9/kube-rbac-proxy/0.log"
Jun 04 00:05:37.263138 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:37.263105    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/2.log"
Jun 04 00:05:37.266500 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:37.266473    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85bd8fd97d-tsn6l_397370f1-b682-48ca-b643-f447f300c2a5/console-operator/3.log"
Jun 04 00:05:37.828194 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:37.828168    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-587cd6c99b-9q5cg_84ec87ca-9194-42d0-a328-0c0e277389ff/console/0.log"
Jun 04 00:05:37.860743 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:37.860721    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-56b467f5ff-994kl_f15e8ee0-46c6-407f-a2c1-a531179f2a9e/download-server/0.log"
Jun 04 00:05:38.529696 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:38.529669    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78dd696465-pk2xd_0a55c225-4fd3-4caa-b9c8-1f535f65637f/volume-data-source-validator/0.log"
Jun 04 00:05:38.854074 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:38.854026    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"]
Jun 04 00:05:38.856648 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:38.856627    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:38.859114 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:38.859091    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w6rbj\"/\"kube-root-ca.crt\""
Jun 04 00:05:38.859789 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:38.859752    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w6rbj\"/\"openshift-service-ca.crt\""
Jun 04 00:05:38.859903 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:38.859755    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-w6rbj\"/\"default-dockercfg-nw9sd\""
Jun 04 00:05:38.868637 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:38.868612    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"]
Jun 04 00:05:39.002736 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.002704    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sgsg\" (UniqueName: \"kubernetes.io/projected/c42663f4-5d66-4680-8138-591d9d88fdff-kube-api-access-5sgsg\") pod \"perf-node-gather-daemonset-m67xp\" (UID: \"c42663f4-5d66-4680-8138-591d9d88fdff\") " pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:39.002736 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.002741    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c42663f4-5d66-4680-8138-591d9d88fdff-proc\") pod \"perf-node-gather-daemonset-m67xp\" (UID: \"c42663f4-5d66-4680-8138-591d9d88fdff\") " pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:39.002967 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.002761    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c42663f4-5d66-4680-8138-591d9d88fdff-lib-modules\") pod \"perf-node-gather-daemonset-m67xp\" (UID: \"c42663f4-5d66-4680-8138-591d9d88fdff\") " pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:39.002967 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.002813    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c42663f4-5d66-4680-8138-591d9d88fdff-sys\") pod \"perf-node-gather-daemonset-m67xp\" (UID: \"c42663f4-5d66-4680-8138-591d9d88fdff\") " pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:39.002967 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.002850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c42663f4-5d66-4680-8138-591d9d88fdff-podres\") pod \"perf-node-gather-daemonset-m67xp\" (UID: \"c42663f4-5d66-4680-8138-591d9d88fdff\") " pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:39.103939 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.103908    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c42663f4-5d66-4680-8138-591d9d88fdff-proc\") pod \"perf-node-gather-daemonset-m67xp\" (UID: \"c42663f4-5d66-4680-8138-591d9d88fdff\") " pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:39.104167 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.103954    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c42663f4-5d66-4680-8138-591d9d88fdff-lib-modules\") pod \"perf-node-gather-daemonset-m67xp\" (UID: \"c42663f4-5d66-4680-8138-591d9d88fdff\") " pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:39.104167 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.103992    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c42663f4-5d66-4680-8138-591d9d88fdff-sys\") pod \"perf-node-gather-daemonset-m67xp\" (UID: \"c42663f4-5d66-4680-8138-591d9d88fdff\") " pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:39.104167 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.104026    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c42663f4-5d66-4680-8138-591d9d88fdff-podres\") pod \"perf-node-gather-daemonset-m67xp\" (UID: \"c42663f4-5d66-4680-8138-591d9d88fdff\") " pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:39.104167 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.104026    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c42663f4-5d66-4680-8138-591d9d88fdff-proc\") pod \"perf-node-gather-daemonset-m67xp\" (UID: \"c42663f4-5d66-4680-8138-591d9d88fdff\") " pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:39.104167 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.104111    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sgsg\" (UniqueName: \"kubernetes.io/projected/c42663f4-5d66-4680-8138-591d9d88fdff-kube-api-access-5sgsg\") pod \"perf-node-gather-daemonset-m67xp\" (UID: \"c42663f4-5d66-4680-8138-591d9d88fdff\") " pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:39.104167 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.104157    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c42663f4-5d66-4680-8138-591d9d88fdff-podres\") pod \"perf-node-gather-daemonset-m67xp\" (UID: \"c42663f4-5d66-4680-8138-591d9d88fdff\") " pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:39.104510 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.104160    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c42663f4-5d66-4680-8138-591d9d88fdff-sys\") pod \"perf-node-gather-daemonset-m67xp\" (UID: \"c42663f4-5d66-4680-8138-591d9d88fdff\") " pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:39.104510 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.104157    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c42663f4-5d66-4680-8138-591d9d88fdff-lib-modules\") pod \"perf-node-gather-daemonset-m67xp\" (UID: \"c42663f4-5d66-4680-8138-591d9d88fdff\") " pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:39.113450 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.113433    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sgsg\" (UniqueName: \"kubernetes.io/projected/c42663f4-5d66-4680-8138-591d9d88fdff-kube-api-access-5sgsg\") pod \"perf-node-gather-daemonset-m67xp\" (UID: \"c42663f4-5d66-4680-8138-591d9d88fdff\") " pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:39.168409 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.168372    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:39.300472 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.300447    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"]
Jun 04 00:05:39.301866 ip-10-0-129-110 kubenswrapper[2577]: W0604 00:05:39.301843    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc42663f4_5d66_4680_8138_591d9d88fdff.slice/crio-48ea714cff8fd70a6d593f8a8b3f8b844f7913c5f6cc8ac42afdf9abf2803a4e WatchSource:0}: Error finding container 48ea714cff8fd70a6d593f8a8b3f8b844f7913c5f6cc8ac42afdf9abf2803a4e: Status 404 returned error can't find the container with id 48ea714cff8fd70a6d593f8a8b3f8b844f7913c5f6cc8ac42afdf9abf2803a4e
Jun 04 00:05:39.946879 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.946852    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5q42g_3a1a6edf-938e-4784-bbc6-74c8829977a2/dns/0.log"
Jun 04 00:05:39.970516 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:39.970477    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5q42g_3a1a6edf-938e-4784-bbc6-74c8829977a2/kube-rbac-proxy/0.log"
Jun 04 00:05:40.046548 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:40.046523    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp" event={"ID":"c42663f4-5d66-4680-8138-591d9d88fdff","Type":"ContainerStarted","Data":"6d0e80000f8a2172edebe36afb08abcea38694175fc239bd3338eb91eb8e395c"}
Jun 04 00:05:40.046690 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:40.046605    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:40.046690 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:40.046623    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp" event={"ID":"c42663f4-5d66-4680-8138-591d9d88fdff","Type":"ContainerStarted","Data":"48ea714cff8fd70a6d593f8a8b3f8b844f7913c5f6cc8ac42afdf9abf2803a4e"}
Jun 04 00:05:40.067712 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:40.067669    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp" podStartSLOduration=2.067656999 podStartE2EDuration="2.067656999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-04 00:05:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-04 00:05:40.066093278 +0000 UTC m=+2462.228771350" watchObservedRunningTime="2026-06-04 00:05:40.067656999 +0000 UTC m=+2462.230335077"
Jun 04 00:05:40.170969 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:40.170945    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n99j6_e42566fd-c3a2-48b7-8d03-a172d908aeff/dns-node-resolver/0.log"
Jun 04 00:05:40.800879 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:40.800841    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6n76l_b70ba3a5-03df-4847-9f88-12c8ae56099c/node-ca/0.log"
Jun 04 00:05:41.926820 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:41.926792    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57684f659d-2qfpm_3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2/kube-auth-proxy/0.log"
Jun 04 00:05:42.068295 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:42.068268    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-l4xgg_11710a07-8ac0-478a-962b-11a2ae70d788/payload-pre-processing/0.log"
Jun 04 00:05:42.125004 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:42.124976    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-sw5zw_5e0dcea1-2a68-4857-aa48-6d21a574b4ca/payload-processing/0.log"
Jun 04 00:05:42.151549 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:42.151528    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55b9b48c7c-jw6cm_98372747-9a9e-4a1e-ad2f-d274d1db6889/router/0.log"
Jun 04 00:05:42.809596 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:42.809570    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-28f9z_201466a6-58e5-49a6-869f-2161d2c8b708/serve-healthcheck-canary/0.log"
Jun 04 00:05:43.449910 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:43.449881    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wwfjd_ae38cd56-f7e1-44da-84b7-90f45b54b423/kube-rbac-proxy/0.log"
Jun 04 00:05:43.469438 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:43.469413    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wwfjd_ae38cd56-f7e1-44da-84b7-90f45b54b423/exporter/0.log"
Jun 04 00:05:43.503154 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:43.503135    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wwfjd_ae38cd56-f7e1-44da-84b7-90f45b54b423/extractor/0.log"
Jun 04 00:05:46.051952 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:46.051918    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-85945fb4bf-qwmr9_3a73e73c-7cc2-4cef-979c-4d214eb7f745/maas-api/0.log"
Jun 04 00:05:46.060942 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:46.060918    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-w6rbj/perf-node-gather-daemonset-m67xp"
Jun 04 00:05:46.168924 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:46.168894    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-85788f9b67-5n46c_79f8e291-c990-4ac4-86b0-6346da1c442c/manager/0.log"
Jun 04 00:05:46.394116 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:46.394085    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f87df8b57-lsfmk_966fd4b6-a8b2-4b59-b62b-a723ee509b6d/manager/0.log"
Jun 04 00:05:46.410926 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:46.410906    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f87df8b57-lsfmk_966fd4b6-a8b2-4b59-b62b-a723ee509b6d/copy-manifests/0.log"
Jun 04 00:05:48.165064 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:48.164990    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6845fb6b5-7rc4p_aa22b043-e050-418b-8628-fc777eb4e306/manager/0.log"
Jun 04 00:05:52.301597 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:52.301562    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57684f659d-2qfpm_3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2/kube-auth-proxy/0.log"
Jun 04 00:05:52.422658 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:52.422612    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-l4xgg_11710a07-8ac0-478a-962b-11a2ae70d788/payload-pre-processing/0.log"
Jun 04 00:05:52.474823 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:52.474792    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-sw5zw_5e0dcea1-2a68-4857-aa48-6d21a574b4ca/payload-processing/0.log"
Jun 04 00:05:52.485809 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:52.485789    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55b9b48c7c-jw6cm_98372747-9a9e-4a1e-ad2f-d274d1db6889/router/0.log"
Jun 04 00:05:53.725927 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:53.725900    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57684f659d-2qfpm_3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2/kube-auth-proxy/0.log"
Jun 04 00:05:53.842167 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:53.842133    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-l4xgg_11710a07-8ac0-478a-962b-11a2ae70d788/payload-pre-processing/0.log"
Jun 04 00:05:53.896861 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:53.896833    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-sw5zw_5e0dcea1-2a68-4857-aa48-6d21a574b4ca/payload-processing/0.log"
Jun 04 00:05:53.909458 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:53.909435    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55b9b48c7c-jw6cm_98372747-9a9e-4a1e-ad2f-d274d1db6889/router/0.log"
Jun 04 00:05:55.558797 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:55.558767    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rgddf_29474ba6-d997-4d27-a609-7adc008c79ce/kube-multus-additional-cni-plugins/0.log"
Jun 04 00:05:55.578567 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:55.578547    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rgddf_29474ba6-d997-4d27-a609-7adc008c79ce/egress-router-binary-copy/0.log"
Jun 04 00:05:55.597488 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:55.597468    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rgddf_29474ba6-d997-4d27-a609-7adc008c79ce/cni-plugins/0.log"
Jun 04 00:05:55.617350 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:55.617330    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rgddf_29474ba6-d997-4d27-a609-7adc008c79ce/bond-cni-plugin/0.log"
Jun 04 00:05:55.638811 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:55.638791    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rgddf_29474ba6-d997-4d27-a609-7adc008c79ce/routeoverride-cni/0.log"
Jun 04 00:05:55.658894 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:55.658875    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rgddf_29474ba6-d997-4d27-a609-7adc008c79ce/whereabouts-cni-bincopy/0.log"
Jun 04 00:05:55.680675 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:55.680653    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rgddf_29474ba6-d997-4d27-a609-7adc008c79ce/whereabouts-cni/0.log"
Jun 04 00:05:55.698264 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:55.698243    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-57684f659d-2qfpm_3e12d177-e8b3-4f63-a3d7-90c9ac28cfc2/kube-auth-proxy/0.log"
Jun 04 00:05:55.814668 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:55.814633    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-l4xgg_11710a07-8ac0-478a-962b-11a2ae70d788/payload-pre-processing/0.log"
Jun 04 00:05:55.864220 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:55.864189    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-sw5zw_5e0dcea1-2a68-4857-aa48-6d21a574b4ca/payload-processing/0.log"
Jun 04 00:05:55.876871 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:55.876851    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55b9b48c7c-jw6cm_98372747-9a9e-4a1e-ad2f-d274d1db6889/router/0.log"
Jun 04 00:05:55.932418 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:55.932396    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rkg7d_06e69283-7ec4-4312-8f28-dc75d920d672/kube-multus/0.log"
Jun 04 00:05:56.037947 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:56.037919    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ssbpc_81ee1ff9-1018-4eaf-b246-18b5f7a95f78/network-metrics-daemon/0.log"
Jun 04 00:05:56.055359 ip-10-0-129-110 kubenswrapper[2577]: I0604 00:05:56.055338    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ssbpc_81ee1ff9-1018-4eaf-b246-18b5f7a95f78/kube-rbac-proxy/0.log"