Jul 10 00:13:40.112969 ip-10-0-128-116 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 10 00:13:40.112985 ip-10-0-128-116 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 10 00:13:40.112996 ip-10-0-128-116 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 10 00:13:40.113364 ip-10-0-128-116 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 10 00:13:50.221668 ip-10-0-128-116 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 10 00:13:50.221687 ip-10-0-128-116 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 43225263a06f46dc89bc0cb95b435f9c --
Jul 10 00:15:50.183689 ip-10-0-128-116 systemd[1]: Starting Kubernetes Kubelet...
Jul 10 00:15:50.572091 ip-10-0-128-116 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 00:15:50.572091 ip-10-0-128-116 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 10 00:15:50.572091 ip-10-0-128-116 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 00:15:50.572091 ip-10-0-128-116 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 10 00:15:50.572091 ip-10-0-128-116 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 00:15:50.573254 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.573098    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 10 00:15:50.580812 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580785    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 00:15:50.580812 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580806    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 00:15:50.580812 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580814    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 00:15:50.580812 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580818    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 00:15:50.580812 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580821    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580825    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580829    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580832    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580835    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580838    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580841    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580845    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580848    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580852    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580855    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580861    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580864    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580867    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580870    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580873    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580876    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580879    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580882    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580887    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 00:15:50.581020 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580890    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580893    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580896    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580899    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580904    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580907    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580910    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580913    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580915    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580919    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580921    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580924    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580928    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580930    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580935    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580938    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580941    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580946    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580949    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580952    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 00:15:50.581536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580955    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580958    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580961    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580964    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580967    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580970    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580973    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580976    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580979    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580984    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580989    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580993    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.580999    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581002    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581005    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581011    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581015    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581019    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581022    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 00:15:50.582078 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581026    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581029    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581035    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581038    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581041    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581045    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581048    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581052    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581057    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581060    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581066    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581069    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581073    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581076    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581082    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581085    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581088    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581091    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581095    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581098    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 00:15:50.582575 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581101    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581105    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581108    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581111    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581114    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581117    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581120    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581126    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581129    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581132    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581137    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581142    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581146    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581149    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581152    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.581155    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582447    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582457    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582460    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582465    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 00:15:50.583086 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582468    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582478    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582481    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582484    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582490    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582493    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582496    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582499    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582502    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582505    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582508    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582511    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582514    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582517    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582520    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582523    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582526    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582532    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582535    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582538    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 00:15:50.583628 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582541    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582543    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582547    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582552    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582556    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582559    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582563    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582566    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582569    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582574    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582577    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582581    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582585    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582588    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582599    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582603    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582606    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582609    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582612    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 00:15:50.584159 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582621    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582624    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582630    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582633    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582636    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582639    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582642    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582644    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582647    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582650    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582653    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582656    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582659    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582662    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582665    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582671    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582674    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582677    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582679    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582683    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 00:15:50.584717 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582686    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582689    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582692    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582695    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582698    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582701    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582704    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582710    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582719    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582722    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582726    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582730    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582733    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582736    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582739    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582741    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582744    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582747    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582751    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 00:15:50.585234 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582757    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582761    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582766    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582769    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582773    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582776    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582779    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582782    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582785    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582789    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582792    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582795    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582801    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582805    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582808    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582811    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.582814    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583596    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583602    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583606    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 00:15:50.585747 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583609    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583619    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583622    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583625    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583629    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583636    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583639    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583642    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583645    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583648    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583651    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583654    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583657    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583661    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583664    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583667    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583670    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583673    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583678    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583681    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 00:15:50.586258 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583684    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583687    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583690    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583693    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583697    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583702    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583705    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583708    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583711    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583714    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583720    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583723    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583726    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583728    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583737    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583740    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583743    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583746    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583749    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 00:15:50.586782 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583752    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583755    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583759    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583764    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583767    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583770    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583775    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583778    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583781    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583785    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583789    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583793    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583796    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583799    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583804    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583806    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583812    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583815    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583818    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 00:15:50.587265 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583821    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583823    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583826    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583829    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583832    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583835    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583838    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583840    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583843    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583852    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583858    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583861    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583865    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583868    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583871    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583874    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583877    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583881    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583884    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 00:15:50.587780 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583888    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583895    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583901    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583903    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583906    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583909    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583912    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583915    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583920    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583923    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583926    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583929    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583931    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583934    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583939    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583943    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583946    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583948    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.583951    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584385    2559 flags.go:64] FLAG: --address="0.0.0.0"
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584407    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 10 00:15:50.588275 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584419    2559 flags.go:64] FLAG: --anonymous-auth="true"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584427    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584434    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584440    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584447    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584453    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584456    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584460    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584466    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584471    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584475    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584479    2559 flags.go:64] FLAG: --cgroup-root=""
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584483    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584488    2559 flags.go:64] FLAG: --client-ca-file=""
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584493    2559 flags.go:64] FLAG: --cloud-provider="external"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584496    2559 flags.go:64] FLAG: --cluster-dns="[]"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584504    2559 flags.go:64] FLAG: --cluster-domain=""
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584507    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584513    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584518    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584522    2559 flags.go:64] FLAG: --container-log-max-files="5"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584526    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584529    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 10 00:15:50.588825 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584533    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584536    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584539    2559 flags.go:64] FLAG: --contention-profiling="false"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584543    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584546    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584549    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584553    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584557    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584562    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584566    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584569    2559 flags.go:64] FLAG: --enable-load-reader="false"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584573    2559 flags.go:64] FLAG: --enable-server="true"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584576    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584582    2559 flags.go:64] FLAG: --event-burst="100"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584585    2559 flags.go:64] FLAG: --event-qps="50"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584588    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584592    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584595    2559 flags.go:64] FLAG: --eviction-hard=""
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584599    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584602    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584605    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584609    2559 flags.go:64] FLAG: --eviction-soft=""
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584612    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584615    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584620    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 10 00:15:50.589400 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584623    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584626    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584629    2559 flags.go:64] FLAG: --fail-swap-on="true"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584632    2559 flags.go:64] FLAG: --feature-gates=""
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584636    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584640    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584643    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584647    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584650    2559 flags.go:64] FLAG: --healthz-port="10248"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584653    2559 flags.go:64] FLAG: --help="false"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584656    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-128-116.ec2.internal"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584659    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584663    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584666    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584670    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584679    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584682    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584686    2559 flags.go:64] FLAG: --image-service-endpoint=""
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584689    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584692    2559 flags.go:64] FLAG: --kube-api-burst="100"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584696    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584699    2559 flags.go:64] FLAG: --kube-api-qps="50"
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584702    2559 flags.go:64] FLAG: --kube-reserved=""
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584705    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 10 00:15:50.590053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584708    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584712    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584715    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584718    2559 flags.go:64] FLAG: --lock-file=""
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584722    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584725    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584728    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584734    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584745    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584749    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584752    2559 flags.go:64] FLAG: --logging-format="text"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584755    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584759    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584762    2559 flags.go:64] FLAG: --manifest-url=""
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584765    2559 flags.go:64] FLAG: --manifest-url-header=""
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584770    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584773    2559 flags.go:64] FLAG: --max-open-files="1000000"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584778    2559 flags.go:64] FLAG: --max-pods="110"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584781    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584784    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584787    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584790    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584794    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584798    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584802    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 10 00:15:50.590681 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584811    2559 flags.go:64] FLAG: --node-status-max-images="50"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584814    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584818    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584821    2559 flags.go:64] FLAG: --pod-cidr=""
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584824    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584831    2559 flags.go:64] FLAG: --pod-manifest-path=""
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584834    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584837    2559 flags.go:64] FLAG: --pods-per-core="0"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584840    2559 flags.go:64] FLAG: --port="10250"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584844    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584847    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d60341d8ce54fc96"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584850    2559 flags.go:64] FLAG: --qos-reserved=""
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584853    2559 flags.go:64] FLAG: --read-only-port="10255"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584857    2559 flags.go:64] FLAG: --register-node="true"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584860    2559 flags.go:64] FLAG: --register-with-taints=""
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584864    2559 flags.go:64] FLAG: --registry-burst="10"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584873    2559 flags.go:64] FLAG: --registry-qps="5"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584877    2559 flags.go:64] FLAG: --reserved-cpus=""
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584880    2559 flags.go:64] FLAG: --reserved-memory=""
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584884    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584887    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584890    2559 flags.go:64] FLAG: --rotate-certificates="false"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584894    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584896    2559 flags.go:64] FLAG: --runonce="false"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584899    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 10 00:15:50.591339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584903    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584906    2559 flags.go:64] FLAG: --seccomp-default="false"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584909    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584912    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584916    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584919    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584923    2559 flags.go:64] FLAG: --storage-driver-password="root"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584927    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584930    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584933    2559 flags.go:64] FLAG: --storage-driver-user="root"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584936    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584939    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584942    2559 flags.go:64] FLAG: --system-cgroups=""
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584945    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584952    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584955    2559 flags.go:64] FLAG: --tls-cert-file=""
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584958    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584967    2559 flags.go:64] FLAG: --tls-min-version=""
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584971    2559 flags.go:64] FLAG: --tls-private-key-file=""
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584973    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584977    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584980    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584983    2559 flags.go:64] FLAG: --v="2"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584987    2559 flags.go:64] FLAG: --version="false"
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584994    2559 flags.go:64] FLAG: --vmodule=""
Jul 10 00:15:50.592006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.584999    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.585002    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585154    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585159    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585162    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585165    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585168    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585171    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585174    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585177    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585180    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585182    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585185    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585190    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585193    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585196    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585198    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585201    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585204    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585207    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 00:15:50.592946 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585210    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585212    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585215    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585218    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585221    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585223    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585226    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585229    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585232    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585235    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585238    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585241    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585245    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585248    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585251    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585254    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585256    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585259    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585262    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585265    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 00:15:50.593883 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585268    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585270    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585273    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585276    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585280    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585284    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585287    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585289    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585293    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585297    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585301    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585305    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585308    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585311    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585314    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585317    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585320    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585323    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585325    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 00:15:50.594803 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585328    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585331    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585334    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585337    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585340    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585342    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585346    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585349    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585353    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585357    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585360    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585363    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585366    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585368    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585371    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585374    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585377    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585385    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585388    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585391    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 00:15:50.595481 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585410    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585413    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585416    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585420    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585423    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585427    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585430    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585434    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585437    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585440    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585444    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585447    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585450    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585453    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585456    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585458    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585462    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585464    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585467    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585471    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 00:15:50.596205 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585474    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 00:15:50.597188 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.585476    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 00:15:50.597188 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.586301    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 00:15:50.597188 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.594889    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 10 00:15:50.597188 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.594916    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 10 00:15:50.597188 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.595840    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 10 00:15:50.597188 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.595856    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 10 00:15:50.597188 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.595976    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 00:15:50.597188 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.595986    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 00:15:50.597188 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.595993    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 00:15:50.597188 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596001    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 00:15:50.597188 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596006    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 00:15:50.597188 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596012    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 00:15:50.597188 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596017    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 00:15:50.597188 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596022    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596027    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596032    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596038    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596043    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596047    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596052    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596056    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596061    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596068    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596075    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596081    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596086    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596099    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596106    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596112    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596118    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596125    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596131    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 00:15:50.597882 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596136    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596140    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596146    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596151    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596156    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596160    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596165    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596170    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596175    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596180    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596185    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596189    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596194    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596199    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596203    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596207    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596211    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596216    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596221    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596225    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 00:15:50.598414 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596229    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596234    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596239    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596244    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596250    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596256    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596262    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596266    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596271    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596275    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596280    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596287    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596292    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596297    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596302    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596308    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596312    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596317    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596321    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 00:15:50.599311 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596326    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596330    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596335    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596339    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596344    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596349    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596377    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596381    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596386    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596390    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596395    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596399    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596404    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596408    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596413    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596418    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596422    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596427    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596431    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596436    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 00:15:50.600141 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596441    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 00:15:50.600802 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596445    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 00:15:50.600802 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596450    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 00:15:50.600802 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596454    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 00:15:50.600802 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596459    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 00:15:50.600802 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596469    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 00:15:50.600802 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596474    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 00:15:50.600802 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596479    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 00:15:50.600802 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596486    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 00:15:50.600802 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596490    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 00:15:50.600802 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596495    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 00:15:50.600802 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596501    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 00:15:50.600802 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596506    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 00:15:50.600802 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596511    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 00:15:50.600802 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.596519    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 00:15:50.600802 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596767    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596776    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596781    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596786    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596790    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596795    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596800    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596805    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596809    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596816    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596822    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596827    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596833    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596838    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596843    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596849    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596854    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596859    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596864    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 00:15:50.601312 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596868    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596873    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596878    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596882    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596892    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596897    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596902    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596907    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596912    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596917    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596921    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596925    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596930    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596935    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596943    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596948    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596952    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596957    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596961    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596966    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 00:15:50.601849 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596970    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596975    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596979    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596984    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596989    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.596996    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597002    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597008    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597013    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597018    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597023    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597028    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597033    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597037    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597044    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597050    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597055    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597062    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597067    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 00:15:50.602370 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597072    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597077    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597082    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597086    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597091    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597095    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597100    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597105    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597109    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597114    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597119    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597123    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597128    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597132    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597137    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597142    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597147    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597152    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597157    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597161    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 00:15:50.602972 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597166    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597170    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597175    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597180    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597185    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597189    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597193    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597197    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597202    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597206    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597211    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597216    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597222    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597226    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597231    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597235    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597240    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597245    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597249    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597254    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 00:15:50.603515 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:50.597259    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 00:15:50.604094 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.597268    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 00:15:50.604094 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.597513    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 10 00:15:50.604094 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.600829    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 10 00:15:50.604094 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.602599    2559 server.go:1019] "Starting client certificate rotation"
Jul 10 00:15:50.604094 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.602711    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 00:15:50.604094 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.602748    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 00:15:50.605943 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.605923    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 00:15:50.607446 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.607424    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 00:15:50.616140 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.616120    2559 log.go:25] "Validated CRI v1 runtime API"
Jul 10 00:15:50.619416 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.619376    2559 log.go:25] "Validated CRI v1 image API"
Jul 10 00:15:50.621830 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.621810    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 10 00:15:50.624484 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.624464    2559 fs.go:135] Filesystem UUIDs: map[1c1cccef-b6df-456a-b587-ef1742d63366:/dev/nvme0n1p3 41b84d7f-9d16-4201-bafc-ea1ccef6f92c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 10 00:15:50.624533 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.624486    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 10 00:15:50.630541 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.630420    2559 manager.go:216] Machine: {Timestamp:2026-07-10 00:15:50.628924628 +0000 UTC m=+0.331709469 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3184219 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28004505ac5c497dd72750629c7aa2 SystemUUID:ec280045-05ac-5c49-7dd7-2750629c7aa2 BootID:43225263-a06f-46dc-89bc-0cb95b435f9c Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c3:37:73:9e:b3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c3:37:73:9e:b3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4a:ad:95:af:ba:77 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 10 00:15:50.630541 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.630535    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 10 00:15:50.630663 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.630630    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 10 00:15:50.634311 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.634276    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 10 00:15:50.634477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.634314    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-116.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 10 00:15:50.636193 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.636182    2559 topology_manager.go:138] "Creating topology manager with none policy"
Jul 10 00:15:50.636237 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.636196    2559 container_manager_linux.go:309] "Creating device plugin manager"
Jul 10 00:15:50.636237 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.636205    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 00:15:50.636864 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.636852    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 00:15:50.639031 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.639014    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 00:15:50.639077 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.639035    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 10 00:15:50.644085 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.644061    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 00:15:50.644218 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.644208    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 10 00:15:50.644250 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.644242    2559 kubelet.go:486] "Attempting to sync node with API server"
Jul 10 00:15:50.644287 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.644257    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 10 00:15:50.644287 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.644281    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 10 00:15:50.644345 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.644296    2559 kubelet.go:398] "Adding apiserver pod source"
Jul 10 00:15:50.644345 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.644310    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 10 00:15:50.644821 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.644802    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 00:15:50.644870 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.644823    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 00:15:50.647866 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.647847    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 10 00:15:50.651173 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.651157    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 10 00:15:50.651240 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.651190    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 10 00:15:50.651364 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.651354    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 10 00:15:50.651431 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.651369    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 10 00:15:50.651431 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.651375    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 10 00:15:50.651431 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.651381    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 10 00:15:50.651431 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.651406    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 10 00:15:50.651431 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.651413    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 10 00:15:50.652267 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.652257    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 10 00:15:50.652267 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.652266    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 10 00:15:50.652333 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.652274    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 10 00:15:50.652333 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.652280    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 10 00:15:50.652333 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.652289    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 10 00:15:50.652333 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.652298    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 10 00:15:50.653257 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.653244    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 10 00:15:50.653257 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.653255    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 10 00:15:50.656833 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.656813    2559 server.go:1268] "Started kubelet"
Jul 10 00:15:50.656940 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.656875    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 10 00:15:50.657037 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.656918    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 10 00:15:50.657037 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.656992    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 10 00:15:50.657825 ip-10-0-128-116 systemd[1]: Started Kubernetes Kubelet.
Jul 10 00:15:50.658189 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:50.658160    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 10 00:15:50.658279 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:50.658200    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-116.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 10 00:15:50.658365 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.658287    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-116.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 10 00:15:50.658906 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.658826    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 10 00:15:50.660200 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.660183    2559 server.go:310] "Adding debug handlers to kubelet server"
Jul 10 00:15:50.660300 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.660280    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gnvks"
Jul 10 00:15:50.664896 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:50.664860    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 10 00:15:50.667787 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:50.665353    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-116.ec2.internal.18c0c5731e2b5846  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-116.ec2.internal,UID:ip-10-0-128-116.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-116.ec2.internal,},FirstTimestamp:2026-07-10 00:15:50.656784454 +0000 UTC m=+0.359569295,LastTimestamp:2026-07-10 00:15:50.656784454 +0000 UTC m=+0.359569295,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-116.ec2.internal,}"
Jul 10 00:15:50.667884 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.667861    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gnvks"
Jul 10 00:15:50.669118 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.669093    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 10 00:15:50.669118 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.669096    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 10 00:15:50.669361 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:50.669317    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-116.ec2.internal\" not found"
Jul 10 00:15:50.669753 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.669736    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 10 00:15:50.669857 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.669782    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 10 00:15:50.670074 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.669951    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 10 00:15:50.670302 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.670017    2559 factory.go:55] Registering systemd factory
Jul 10 00:15:50.670417 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.670405    2559 factory.go:223] Registration of the systemd container factory successfully
Jul 10 00:15:50.670540 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.670407    2559 reconstruct.go:97] "Volume reconstruction finished"
Jul 10 00:15:50.670618 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.670601    2559 reconciler.go:29] "Reconciler: start to sync state"
Jul 10 00:15:50.670743 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.670730    2559 factory.go:153] Registering CRI-O factory
Jul 10 00:15:50.670743 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.670743    2559 factory.go:223] Registration of the crio container factory successfully
Jul 10 00:15:50.670850 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.670787    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 10 00:15:50.670850 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.670808    2559 factory.go:103] Registering Raw factory
Jul 10 00:15:50.670850 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.670819    2559 manager.go:1196] Started watching for new ooms in manager
Jul 10 00:15:50.672295 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.672277    2559 manager.go:349] Starting recovery of all containers
Jul 10 00:15:50.681490 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.681453    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 00:15:50.684034 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.684019    2559 manager.go:354] Recovery completed
Jul 10 00:15:50.684319 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:50.684299    2559 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-116.ec2.internal\" not found" node="ip-10-0-128-116.ec2.internal"
Jul 10 00:15:50.688577 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.688562    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 00:15:50.690446 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.690429    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 00:15:50.690516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.690461    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 00:15:50.690516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.690474    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasSufficientPID"
Jul 10 00:15:50.691499 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.691481    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 10 00:15:50.691578 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.691500    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 00:15:50.691578 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.691499    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 10 00:15:50.691578 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.691573    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 00:15:50.692221 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.692205    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 00:15:50.692297 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.692236    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 00:15:50.692297 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.692246    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasSufficientPID"
Jul 10 00:15:50.692908 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.692894    2559 policy_none.go:49] "None policy: Start"
Jul 10 00:15:50.692943 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.692913    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 10 00:15:50.692943 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.692925    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 10 00:15:50.693990 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.693979    2559 policy_none.go:47] "Start"
Jul 10 00:15:50.736021 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.736002    2559 manager.go:337] "Starting Device Plugin manager"
Jul 10 00:15:50.736125 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:50.736109    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 10 00:15:50.736161 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.736129    2559 server.go:87] "Starting device plugin registration server"
Jul 10 00:15:50.736463 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.736447    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 10 00:15:50.736566 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.736464    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 10 00:15:50.736566 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.736558    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 10 00:15:50.736726 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.736644    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 10 00:15:50.736726 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.736671    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 10 00:15:50.737414 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:50.737388    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 10 00:15:50.737491 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:50.737446    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-116.ec2.internal\" not found"
Jul 10 00:15:50.828721 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.828634    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 10 00:15:50.829981 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.829962    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 10 00:15:50.830070 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.830006    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 10 00:15:50.830070 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.830031    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 10 00:15:50.830181 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:50.830074    2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 10 00:15:50.832487 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.832467    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 00:15:50.837403 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.837384    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 00:15:50.838236 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.838211    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 00:15:50.838314 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.838246    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 00:15:50.838314 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.838263    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasSufficientPID"
Jul 10 00:15:50.838314 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.838292    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-116.ec2.internal"
Jul 10 00:15:50.846421 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.846405    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-116.ec2.internal"
Jul 10 00:15:50.846476 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:50.846429    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-116.ec2.internal\": node \"ip-10-0-128-116.ec2.internal\" not found"
Jul 10 00:15:50.864308 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:50.864284    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-116.ec2.internal\" not found"
Jul 10 00:15:50.931099 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.931057    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-116.ec2.internal"]
Jul 10 00:15:50.931225 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.931207    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 00:15:50.933047 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.933032    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 00:15:50.933123 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.933065    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 00:15:50.933123 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.933083    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasSufficientPID"
Jul 10 00:15:50.934579 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.934566    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 00:15:50.934748 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.934734    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal"
Jul 10 00:15:50.934828 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.934765    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 00:15:50.935348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.935314    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 00:15:50.935425 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.935359    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 00:15:50.935425 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.935369    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasSufficientPID"
Jul 10 00:15:50.936121 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.936091    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 00:15:50.936209 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.936127    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 00:15:50.936209 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.936142    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasSufficientPID"
Jul 10 00:15:50.937382 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.937366    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-116.ec2.internal"
Jul 10 00:15:50.937485 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.937417    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 00:15:50.938196 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.938173    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 00:15:50.938277 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.938206    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 00:15:50.938277 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.938219    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeHasSufficientPID"
Jul 10 00:15:50.961122 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:50.961101    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-116.ec2.internal\" not found" node="ip-10-0-128-116.ec2.internal"
Jul 10 00:15:50.964405 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:50.964385    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-116.ec2.internal\" not found"
Jul 10 00:15:50.965967 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:50.965952    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-116.ec2.internal\" not found" node="ip-10-0-128-116.ec2.internal"
Jul 10 00:15:50.972476 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.972459    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43b563d1079fd4e3284b95fba6c82049-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal\" (UID: \"43b563d1079fd4e3284b95fba6c82049\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal"
Jul 10 00:15:50.972534 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.972483    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/92d561d32137a202f7c75fa6c2ab0660-config\") pod \"kube-apiserver-proxy-ip-10-0-128-116.ec2.internal\" (UID: \"92d561d32137a202f7c75fa6c2ab0660\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-116.ec2.internal"
Jul 10 00:15:50.972534 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:50.972504    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/43b563d1079fd4e3284b95fba6c82049-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal\" (UID: \"43b563d1079fd4e3284b95fba6c82049\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal"
Jul 10 00:15:51.064965 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:51.064933    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-116.ec2.internal\" not found"
Jul 10 00:15:51.072818 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.072798    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/92d561d32137a202f7c75fa6c2ab0660-config\") pod \"kube-apiserver-proxy-ip-10-0-128-116.ec2.internal\" (UID: \"92d561d32137a202f7c75fa6c2ab0660\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-116.ec2.internal"
Jul 10 00:15:51.072877 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.072831    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/43b563d1079fd4e3284b95fba6c82049-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal\" (UID: \"43b563d1079fd4e3284b95fba6c82049\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal"
Jul 10 00:15:51.072877 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.072855    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43b563d1079fd4e3284b95fba6c82049-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal\" (UID: \"43b563d1079fd4e3284b95fba6c82049\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal"
Jul 10 00:15:51.072947 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.072900    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43b563d1079fd4e3284b95fba6c82049-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal\" (UID: \"43b563d1079fd4e3284b95fba6c82049\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal"
Jul 10 00:15:51.072947 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.072911    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/92d561d32137a202f7c75fa6c2ab0660-config\") pod \"kube-apiserver-proxy-ip-10-0-128-116.ec2.internal\" (UID: \"92d561d32137a202f7c75fa6c2ab0660\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-116.ec2.internal"
Jul 10 00:15:51.073013 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.072915    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/43b563d1079fd4e3284b95fba6c82049-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal\" (UID: \"43b563d1079fd4e3284b95fba6c82049\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal"
Jul 10 00:15:51.166373 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:51.166269    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-116.ec2.internal\" not found"
Jul 10 00:15:51.270618 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:51.270581    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-116.ec2.internal\" not found"
Jul 10 00:15:51.270710 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.270650    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal"
Jul 10 00:15:51.271826 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.271812    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-116.ec2.internal"
Jul 10 00:15:51.372409 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:51.372358    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-116.ec2.internal\" not found"
Jul 10 00:15:51.476158 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:51.476087    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-116.ec2.internal\" not found"
Jul 10 00:15:51.576521 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:51.576479    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-116.ec2.internal\" not found"
Jul 10 00:15:51.602612 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.602576    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 10 00:15:51.602785 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.602763    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 00:15:51.602823 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.602795    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 00:15:51.676950 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:51.676919    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-116.ec2.internal\" not found"
Jul 10 00:15:51.678668 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.678641    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 00:10:51 +0000 UTC" deadline="2027-12-23 15:52:08.261414541 +0000 UTC"
Jul 10 00:15:51.678739 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.678669    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12759h36m16.582748459s"
Jul 10 00:15:51.731126 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.731043    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 00:15:51.777053 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:51.777017    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-116.ec2.internal\" not found"
Jul 10 00:15:51.878064 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:51.878030    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-116.ec2.internal\" not found"
Jul 10 00:15:51.900042 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:51.900004    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92d561d32137a202f7c75fa6c2ab0660.slice/crio-d786083f4ef5f02d19c3b2aba88d5cef7c1d19b5afd9d52835d1a299233d314f WatchSource:0}: Error finding container d786083f4ef5f02d19c3b2aba88d5cef7c1d19b5afd9d52835d1a299233d314f: Status 404 returned error can't find the container with id d786083f4ef5f02d19c3b2aba88d5cef7c1d19b5afd9d52835d1a299233d314f
Jul 10 00:15:51.900290 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:51.900264    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b563d1079fd4e3284b95fba6c82049.slice/crio-3ee2b88037051eb946dea721efedf3626bfb81d1d7e246badc9b0ff4ee6a587e WatchSource:0}: Error finding container 3ee2b88037051eb946dea721efedf3626bfb81d1d7e246badc9b0ff4ee6a587e: Status 404 returned error can't find the container with id 3ee2b88037051eb946dea721efedf3626bfb81d1d7e246badc9b0ff4ee6a587e
Jul 10 00:15:51.904227 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.904208    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 00:15:51.955036 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.955005    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 00:15:51.969438 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.969411    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal"
Jul 10 00:15:51.981764 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.981705    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 00:15:51.982742 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.982728    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-116.ec2.internal"
Jul 10 00:15:51.991753 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:51.991735    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 00:15:52.426886 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.426804    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 00:15:52.646217 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.646161    2559 apiserver.go:52] "Watching apiserver"
Jul 10 00:15:52.656302 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.656277    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 10 00:15:52.656668 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.656644    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-tdzvd","openshift-multus/network-metrics-daemon-qzdmp","kube-system/global-pull-secret-syncer-6x94h","kube-system/kube-apiserver-proxy-ip-10-0-128-116.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal","openshift-network-diagnostics/network-check-target-klmj6","openshift-network-operator/iptables-alerter-9l7jh","openshift-ovn-kubernetes/ovnkube-node-d2s7q","kube-system/konnectivity-agent-cldmd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8","openshift-cluster-node-tuning-operator/tuned-g88v9","openshift-dns/node-resolver-wtmzf","openshift-image-registry/node-ca-gphtp","openshift-multus/multus-529b2"]
Jul 10 00:15:52.659188 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.659168    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.660470 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.660454    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.661598 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.661580    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:15:52.661688 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:52.661643    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:15:52.661947 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.661930    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 10 00:15:52.662131 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.662114    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 10 00:15:52.662224 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.662209    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-79x2w\""
Jul 10 00:15:52.662364 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.662349    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 10 00:15:52.662633 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.662619    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.662943 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.662927    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 10 00:15:52.663396 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.663367    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 10 00:15:52.663508 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.663459    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pk5p7\""
Jul 10 00:15:52.664868 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.664832    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.664966 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.664919    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 10 00:15:52.665275 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.665255    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 10 00:15:52.665359 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.665282    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 10 00:15:52.665359 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.665297    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wf5ff\""
Jul 10 00:15:52.665359 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.665257    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 10 00:15:52.668087 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.668071    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:15:52.668155 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:52.668129    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:15:52.668716 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.668699    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 10 00:15:52.668829 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.668730    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 10 00:15:52.668829 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.668766    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 10 00:15:52.668991 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.668826    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 10 00:15:52.668991 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.668831    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9lc9q\""
Jul 10 00:15:52.668991 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.668827    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 10 00:15:52.669148 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.669086    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 10 00:15:52.669469 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.669456    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cldmd"
Jul 10 00:15:52.669570 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.669461    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 10 00:15:52.670653 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.670636    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gphtp"
Jul 10 00:15:52.672006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.671984    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:15:52.672097 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:52.672070    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:15:52.672164 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.672125    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 10 00:15:52.672289 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.672263    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 10 00:15:52.672820 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.672804    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c29g4\""
Jul 10 00:15:52.673092 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.673077    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 10 00:15:52.673446 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.673424    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 10 00:15:52.673546 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.673520    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qs4pw\""
Jul 10 00:15:52.673612 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.673430    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 10 00:15:52.673738 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.673723    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.674910 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.674894    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9l7jh"
Jul 10 00:15:52.676214 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.676195    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wtmzf"
Jul 10 00:15:52.676992 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.676719    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 10 00:15:52.676992 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.676735    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-t5l8d\""
Jul 10 00:15:52.676992 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.676894    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 10 00:15:52.679993 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.678324    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qsvl8\""
Jul 10 00:15:52.679993 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.678379    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 10 00:15:52.679993 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.678613    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 10 00:15:52.679993 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.678845    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 10 00:15:52.679993 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.679031    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 10 00:15:52.679993 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.679311    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tp8qp\""
Jul 10 00:15:52.680482 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.680460    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 10 00:15:52.681851 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.681829    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-registration-dir\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.681934 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.681866    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-systemd\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.681934 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.681892    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p57kj\" (UniqueName: \"kubernetes.io/projected/c6831fcd-2a29-48d5-aa85-b06e836645c6-kube-api-access-p57kj\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.682048 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.681962    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ce01e87-f4a3-4774-b800-f1dc965f9689-kubelet-config\") pod \"global-pull-secret-syncer-6x94h\" (UID: \"7ce01e87-f4a3-4774-b800-f1dc965f9689\") " pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:15:52.682048 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682002    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-os-release\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.682048 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682030    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-var-lib-cni-multus\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.682211 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682055    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f75ad69a-e4b3-484a-a93c-468bd16e460c-env-overrides\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.682211 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682080    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-hostroot\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.682211 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682103    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-etc-kubernetes\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.682211 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682137    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-kubelet\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.682211 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682158    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-run-systemd\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.682211 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682173    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f75ad69a-e4b3-484a-a93c-468bd16e460c-ovnkube-config\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.682211 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682195    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f75ad69a-e4b3-484a-a93c-468bd16e460c-ovn-node-metrics-cert\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.682636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682238    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.682636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682267    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-sysconfig\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.682636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682287    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-lib-modules\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.682636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682335    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-slash\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.682636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682358    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-cni-bin\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.682636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682384    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/615763a8-ff81-4ac2-af5a-0b3c7b016222-konnectivity-ca\") pod \"konnectivity-agent-cldmd\" (UID: \"615763a8-ff81-4ac2-af5a-0b3c7b016222\") " pod="kube-system/konnectivity-agent-cldmd"
Jul 10 00:15:52.682636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682422    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg68d\" (UniqueName: \"kubernetes.io/projected/fc6182be-be7a-402a-b5b0-584a1e38e26d-kube-api-access-lg68d\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.682636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682445    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-run\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.682636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682464    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret\") pod \"global-pull-secret-syncer-6x94h\" (UID: \"7ce01e87-f4a3-4774-b800-f1dc965f9689\") " pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:15:52.682636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682480    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-cni-netd\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.682636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682496    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7ba31ebb-c51d-4692-80de-d8a7384b7e2c-host\") pod \"node-ca-gphtp\" (UID: \"7ba31ebb-c51d-4692-80de-d8a7384b7e2c\") " pod="openshift-image-registry/node-ca-gphtp"
Jul 10 00:15:52.682636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682510    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-system-cni-dir\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.682636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682537    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-cni-binary-copy\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.682636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682556    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-run-ovn-kubernetes\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.682636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682581    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f75ad69a-e4b3-484a-a93c-468bd16e460c-ovnkube-script-lib\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.682636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682623    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpq69\" (UniqueName: \"kubernetes.io/projected/7ba31ebb-c51d-4692-80de-d8a7384b7e2c-kube-api-access-zpq69\") pod \"node-ca-gphtp\" (UID: \"7ba31ebb-c51d-4692-80de-d8a7384b7e2c\") " pod="openshift-image-registry/node-ca-gphtp"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682659    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-sysctl-d\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682691    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-var-lib-kubelet\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682719    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-multus-conf-dir\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682741    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-multus-daemon-config\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682773    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7ba31ebb-c51d-4692-80de-d8a7384b7e2c-serviceca\") pod \"node-ca-gphtp\" (UID: \"7ba31ebb-c51d-4692-80de-d8a7384b7e2c\") " pod="openshift-image-registry/node-ca-gphtp"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682798    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-etc-selinux\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682814    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-multus-cni-dir\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682856    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-run-k8s-cni-cncf-io\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682878    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ce01e87-f4a3-4774-b800-f1dc965f9689-dbus\") pod \"global-pull-secret-syncer-6x94h\" (UID: \"7ce01e87-f4a3-4774-b800-f1dc965f9689\") " pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682893    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-run-netns\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682926    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-run-ovn\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682949    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-node-log\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682964    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-socket-dir\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.682979    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-sys-fs\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683000    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-modprobe-d\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683021    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-sys\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.683462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683036    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6831fcd-2a29-48d5-aa85-b06e836645c6-tmp\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.684324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683050    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6745\" (UniqueName: \"kubernetes.io/projected/f75ad69a-e4b3-484a-a93c-468bd16e460c-kube-api-access-k6745\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.684324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683065    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l97n\" (UniqueName: \"kubernetes.io/projected/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-kube-api-access-2l97n\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.684324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683093    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.684324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683114    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/615763a8-ff81-4ac2-af5a-0b3c7b016222-agent-certs\") pod \"konnectivity-agent-cldmd\" (UID: \"615763a8-ff81-4ac2-af5a-0b3c7b016222\") " pod="kube-system/konnectivity-agent-cldmd"
Jul 10 00:15:52.684324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683130    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-var-lib-kubelet\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.684324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683144    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-tuned\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.684324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683163    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-var-lib-openvswitch\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.684324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683202    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfdcq\" (UniqueName: \"kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq\") pod \"network-check-target-klmj6\" (UID: \"f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55\") " pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:15:52.684324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683220    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-host\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.684324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683235    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-var-lib-cni-bin\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.684324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683251    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-run-openvswitch\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.684324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683275    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-device-dir\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.684324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683289    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-kubernetes\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.684324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683339    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-cnibin\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.684324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683365    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-systemd-units\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.684324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683381    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-run-netns\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.685166 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683396    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-multus-socket-dir-parent\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.685166 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683420    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-run-multus-certs\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.685166 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683446    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-etc-openvswitch\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.685166 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683467    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-log-socket\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.685166 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.683482    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-sysctl-conf\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.685166 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.684917    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 00:15:52.706842 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.706823    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-c79mg"
Jul 10 00:15:52.715215 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.715193    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-c79mg"
Jul 10 00:15:52.770632 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.770607    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 10 00:15:52.784411 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784391    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-systemd-units\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.784492 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784418    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-run-netns\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.784492 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784447    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e12398d-7b1b-478b-8031-8b0e7617091d-system-cni-dir\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.784492 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784469    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b9f0bd1-9d79-4814-ae7c-050412fdc4c6-host-slash\") pod \"iptables-alerter-9l7jh\" (UID: \"5b9f0bd1-9d79-4814-ae7c-050412fdc4c6\") " pod="openshift-network-operator/iptables-alerter-9l7jh"
Jul 10 00:15:52.784593 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784495    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-multus-socket-dir-parent\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.784593 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784507    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-systemd-units\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.784593 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784522    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-run-multus-certs\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.784593 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784524    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-run-netns\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.784593 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784565    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-multus-socket-dir-parent\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.784737 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784608    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-etc-openvswitch\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.784737 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784619    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-run-multus-certs\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.784737 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784668    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-etc-openvswitch\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.784737 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784657    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-log-socket\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.784737 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784700    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccctr\" (UniqueName: \"kubernetes.io/projected/5b9f0bd1-9d79-4814-ae7c-050412fdc4c6-kube-api-access-ccctr\") pod \"iptables-alerter-9l7jh\" (UID: \"5b9f0bd1-9d79-4814-ae7c-050412fdc4c6\") " pod="openshift-network-operator/iptables-alerter-9l7jh"
Jul 10 00:15:52.784737 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784703    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-log-socket\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.784737 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784720    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-sysctl-conf\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.784948 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784750    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e12398d-7b1b-478b-8031-8b0e7617091d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.784948 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784785    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-registration-dir\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.784948 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784809    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-systemd\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.784948 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784825    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p57kj\" (UniqueName: \"kubernetes.io/projected/c6831fcd-2a29-48d5-aa85-b06e836645c6-kube-api-access-p57kj\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.784948 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784827    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-sysctl-conf\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.784948 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784843    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ce01e87-f4a3-4774-b800-f1dc965f9689-kubelet-config\") pod \"global-pull-secret-syncer-6x94h\" (UID: \"7ce01e87-f4a3-4774-b800-f1dc965f9689\") " pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:15:52.784948 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784846    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-registration-dir\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.784948 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784862    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-systemd\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.784948 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784861    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-os-release\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.784948 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784899    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-var-lib-cni-multus\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.784948 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784925    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f75ad69a-e4b3-484a-a93c-468bd16e460c-env-overrides\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.784948 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784929    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ce01e87-f4a3-4774-b800-f1dc965f9689-kubelet-config\") pod \"global-pull-secret-syncer-6x94h\" (UID: \"7ce01e87-f4a3-4774-b800-f1dc965f9689\") " pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784965    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e12398d-7b1b-478b-8031-8b0e7617091d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784971    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-os-release\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784981    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-var-lib-cni-multus\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.784993    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-hostroot\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785021    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-hostroot\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785042    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-etc-kubernetes\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785066    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-kubelet\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785090    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-run-systemd\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785122    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f75ad69a-e4b3-484a-a93c-468bd16e460c-ovnkube-config\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785138    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-etc-kubernetes\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785142    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f75ad69a-e4b3-484a-a93c-468bd16e460c-ovn-node-metrics-cert\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785166    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-run-systemd\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785144    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-kubelet\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785182    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785209    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-sysconfig\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785233    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-lib-modules\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785261    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.785477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785255    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-slash\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785294    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-sysconfig\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785314    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-cni-bin\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785360    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/615763a8-ff81-4ac2-af5a-0b3c7b016222-konnectivity-ca\") pod \"konnectivity-agent-cldmd\" (UID: \"615763a8-ff81-4ac2-af5a-0b3c7b016222\") " pod="kube-system/konnectivity-agent-cldmd"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785377    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-slash\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785387    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg68d\" (UniqueName: \"kubernetes.io/projected/fc6182be-be7a-402a-b5b0-584a1e38e26d-kube-api-access-lg68d\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785414    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-run\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785431    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-cni-bin\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785462    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret\") pod \"global-pull-secret-syncer-6x94h\" (UID: \"7ce01e87-f4a3-4774-b800-f1dc965f9689\") " pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785489    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-cni-netd\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785501    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f75ad69a-e4b3-484a-a93c-468bd16e460c-env-overrides\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785500    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-run\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785511    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785415    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-lib-modules\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785559    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7ba31ebb-c51d-4692-80de-d8a7384b7e2c-host\") pod \"node-ca-gphtp\" (UID: \"7ba31ebb-c51d-4692-80de-d8a7384b7e2c\") " pod="openshift-image-registry/node-ca-gphtp"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785573    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-cni-netd\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:52.785591    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785518    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7ba31ebb-c51d-4692-80de-d8a7384b7e2c-host\") pod \"node-ca-gphtp\" (UID: \"7ba31ebb-c51d-4692-80de-d8a7384b7e2c\") " pod="openshift-image-registry/node-ca-gphtp"
Jul 10 00:15:52.786348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785635    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5b9f0bd1-9d79-4814-ae7c-050412fdc4c6-iptables-alerter-script\") pod \"iptables-alerter-9l7jh\" (UID: \"5b9f0bd1-9d79-4814-ae7c-050412fdc4c6\") " pod="openshift-network-operator/iptables-alerter-9l7jh"
Jul 10 00:15:52.787156 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:52.785665    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret podName:7ce01e87-f4a3-4774-b800-f1dc965f9689 nodeName:}" failed. No retries permitted until 2026-07-10 00:15:53.285642012 +0000 UTC m=+2.988426861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret") pod "global-pull-secret-syncer-6x94h" (UID: "7ce01e87-f4a3-4774-b800-f1dc965f9689") : object "kube-system"/"original-pull-secret" not registered
Jul 10 00:15:52.787156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785698    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-system-cni-dir\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.787156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785732    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f75ad69a-e4b3-484a-a93c-468bd16e460c-ovnkube-config\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.787156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785734    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-cni-binary-copy\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.787156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785777    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-run-ovn-kubernetes\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.787156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785813    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-system-cni-dir\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.787156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785830    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f75ad69a-e4b3-484a-a93c-468bd16e460c-ovnkube-script-lib\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.787156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785859    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpq69\" (UniqueName: \"kubernetes.io/projected/7ba31ebb-c51d-4692-80de-d8a7384b7e2c-kube-api-access-zpq69\") pod \"node-ca-gphtp\" (UID: \"7ba31ebb-c51d-4692-80de-d8a7384b7e2c\") " pod="openshift-image-registry/node-ca-gphtp"
Jul 10 00:15:52.787156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785865    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-run-ovn-kubernetes\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.787156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785886    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e12398d-7b1b-478b-8031-8b0e7617091d-cni-binary-copy\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.787156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785912    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26bc4\" (UniqueName: \"kubernetes.io/projected/6e12398d-7b1b-478b-8031-8b0e7617091d-kube-api-access-26bc4\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.787156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785955    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmfmc\" (UniqueName: \"kubernetes.io/projected/bc636ec8-6508-407c-8f57-38cf0bcf86a3-kube-api-access-rmfmc\") pod \"node-resolver-wtmzf\" (UID: \"bc636ec8-6508-407c-8f57-38cf0bcf86a3\") " pod="openshift-dns/node-resolver-wtmzf"
Jul 10 00:15:52.787156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785961    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/615763a8-ff81-4ac2-af5a-0b3c7b016222-konnectivity-ca\") pod \"konnectivity-agent-cldmd\" (UID: \"615763a8-ff81-4ac2-af5a-0b3c7b016222\") " pod="kube-system/konnectivity-agent-cldmd"
Jul 10 00:15:52.787156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.785997    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-sysctl-d\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.787156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786029    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-var-lib-kubelet\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.787156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786055    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-multus-conf-dir\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786081    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-multus-daemon-config\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786116    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7ba31ebb-c51d-4692-80de-d8a7384b7e2c-serviceca\") pod \"node-ca-gphtp\" (UID: \"7ba31ebb-c51d-4692-80de-d8a7384b7e2c\") " pod="openshift-image-registry/node-ca-gphtp"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786135    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-var-lib-kubelet\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786143    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e12398d-7b1b-478b-8031-8b0e7617091d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786185    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-sysctl-d\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786186    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2rlj\" (UniqueName: \"kubernetes.io/projected/1535944a-df71-496e-955d-7d67d827bec0-kube-api-access-v2rlj\") pod \"network-metrics-daemon-qzdmp\" (UID: \"1535944a-df71-496e-955d-7d67d827bec0\") " pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786225    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-multus-conf-dir\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786228    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-etc-selinux\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786286    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-multus-cni-dir\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786330    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-cni-binary-copy\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786312    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-run-k8s-cni-cncf-io\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786394    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-multus-cni-dir\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786403    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f75ad69a-e4b3-484a-a93c-468bd16e460c-ovnkube-script-lib\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786411    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-etc-selinux\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786406    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-run-k8s-cni-cncf-io\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786431    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e12398d-7b1b-478b-8031-8b0e7617091d-os-release\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.788018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786488    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ce01e87-f4a3-4774-b800-f1dc965f9689-dbus\") pod \"global-pull-secret-syncer-6x94h\" (UID: \"7ce01e87-f4a3-4774-b800-f1dc965f9689\") " pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786524    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-run-netns\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786553    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-run-ovn\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786564    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-run-netns\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786578    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-node-log\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786596    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ce01e87-f4a3-4774-b800-f1dc965f9689-dbus\") pod \"global-pull-secret-syncer-6x94h\" (UID: \"7ce01e87-f4a3-4774-b800-f1dc965f9689\") " pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786614    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e12398d-7b1b-478b-8031-8b0e7617091d-cnibin\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786622    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-node-log\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786615    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-run-ovn\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786642    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-socket-dir\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786667    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-sys-fs\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786682    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-multus-daemon-config\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786695    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7ba31ebb-c51d-4692-80de-d8a7384b7e2c-serviceca\") pod \"node-ca-gphtp\" (UID: \"7ba31ebb-c51d-4692-80de-d8a7384b7e2c\") " pod="openshift-image-registry/node-ca-gphtp"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786702    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-modprobe-d\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786726    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-sys-fs\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786727    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-sys\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786741    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-socket-dir\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786765    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6831fcd-2a29-48d5-aa85-b06e836645c6-tmp\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.788962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786774    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-modprobe-d\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786768    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-sys\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786790    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6745\" (UniqueName: \"kubernetes.io/projected/f75ad69a-e4b3-484a-a93c-468bd16e460c-kube-api-access-k6745\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786820    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs\") pod \"network-metrics-daemon-qzdmp\" (UID: \"1535944a-df71-496e-955d-7d67d827bec0\") " pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786845    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l97n\" (UniqueName: \"kubernetes.io/projected/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-kube-api-access-2l97n\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786862    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786887    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/615763a8-ff81-4ac2-af5a-0b3c7b016222-agent-certs\") pod \"konnectivity-agent-cldmd\" (UID: \"615763a8-ff81-4ac2-af5a-0b3c7b016222\") " pod="kube-system/konnectivity-agent-cldmd"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.786942    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787012    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-var-lib-kubelet\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787037    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-tuned\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787074    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-var-lib-openvswitch\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787092    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfdcq\" (UniqueName: \"kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq\") pod \"network-check-target-klmj6\" (UID: \"f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55\") " pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787110    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-host\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787134    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-var-lib-cni-bin\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787160    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-run-openvswitch\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787154    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-var-lib-kubelet\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787187    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-device-dir\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.789751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787211    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-kubernetes\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.790353 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787232    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-host-var-lib-cni-bin\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.790353 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787241    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bc636ec8-6508-407c-8f57-38cf0bcf86a3-hosts-file\") pod \"node-resolver-wtmzf\" (UID: \"bc636ec8-6508-407c-8f57-38cf0bcf86a3\") " pod="openshift-dns/node-resolver-wtmzf"
Jul 10 00:15:52.790353 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787285    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-host\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.790353 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787313    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc636ec8-6508-407c-8f57-38cf0bcf86a3-tmp-dir\") pod \"node-resolver-wtmzf\" (UID: \"bc636ec8-6508-407c-8f57-38cf0bcf86a3\") " pod="openshift-dns/node-resolver-wtmzf"
Jul 10 00:15:52.790353 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787315    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-run-openvswitch\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.790353 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787374    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-cnibin\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.790353 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787379    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fc6182be-be7a-402a-b5b0-584a1e38e26d-device-dir\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.790353 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787389    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-kubernetes\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.790353 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787492    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-cnibin\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.790353 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.787506    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f75ad69a-e4b3-484a-a93c-468bd16e460c-var-lib-openvswitch\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.790353 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.789207    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6831fcd-2a29-48d5-aa85-b06e836645c6-tmp\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.790353 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.789775    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c6831fcd-2a29-48d5-aa85-b06e836645c6-etc-tuned\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.790353 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.789880    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/615763a8-ff81-4ac2-af5a-0b3c7b016222-agent-certs\") pod \"konnectivity-agent-cldmd\" (UID: \"615763a8-ff81-4ac2-af5a-0b3c7b016222\") " pod="kube-system/konnectivity-agent-cldmd"
Jul 10 00:15:52.794061 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:52.793970    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 00:15:52.794061 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:52.794019    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 00:15:52.794061 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:52.794034    2559 projected.go:196] Error preparing data for projected volume kube-api-access-bfdcq for pod openshift-network-diagnostics/network-check-target-klmj6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 00:15:52.794830 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.794116    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f75ad69a-e4b3-484a-a93c-468bd16e460c-ovn-node-metrics-cert\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.794830 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:52.794177    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq podName:f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55 nodeName:}" failed. No retries permitted until 2026-07-10 00:15:53.294156553 +0000 UTC m=+2.996941397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bfdcq" (UniqueName: "kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq") pod "network-check-target-klmj6" (UID: "f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 00:15:52.794830 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.794594    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p57kj\" (UniqueName: \"kubernetes.io/projected/c6831fcd-2a29-48d5-aa85-b06e836645c6-kube-api-access-p57kj\") pod \"tuned-g88v9\" (UID: \"c6831fcd-2a29-48d5-aa85-b06e836645c6\") " pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.795281 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.795261    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpq69\" (UniqueName: \"kubernetes.io/projected/7ba31ebb-c51d-4692-80de-d8a7384b7e2c-kube-api-access-zpq69\") pod \"node-ca-gphtp\" (UID: \"7ba31ebb-c51d-4692-80de-d8a7384b7e2c\") " pod="openshift-image-registry/node-ca-gphtp"
Jul 10 00:15:52.795650 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.795633    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l97n\" (UniqueName: \"kubernetes.io/projected/1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6-kube-api-access-2l97n\") pod \"multus-529b2\" (UID: \"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6\") " pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.795974 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.795954    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg68d\" (UniqueName: \"kubernetes.io/projected/fc6182be-be7a-402a-b5b0-584a1e38e26d-kube-api-access-lg68d\") pod \"aws-ebs-csi-driver-node-sfxc8\" (UID: \"fc6182be-be7a-402a-b5b0-584a1e38e26d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.796231 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.796214    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6745\" (UniqueName: \"kubernetes.io/projected/f75ad69a-e4b3-484a-a93c-468bd16e460c-kube-api-access-k6745\") pod \"ovnkube-node-d2s7q\" (UID: \"f75ad69a-e4b3-484a-a93c-468bd16e460c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.809447 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.809426    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 00:15:52.834738 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.834685    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal" event={"ID":"43b563d1079fd4e3284b95fba6c82049","Type":"ContainerStarted","Data":"3ee2b88037051eb946dea721efedf3626bfb81d1d7e246badc9b0ff4ee6a587e"}
Jul 10 00:15:52.835802 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.835781    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-116.ec2.internal" event={"ID":"92d561d32137a202f7c75fa6c2ab0660","Type":"ContainerStarted","Data":"d786083f4ef5f02d19c3b2aba88d5cef7c1d19b5afd9d52835d1a299233d314f"}
Jul 10 00:15:52.887884 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.887850    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e12398d-7b1b-478b-8031-8b0e7617091d-cni-binary-copy\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.888005 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.887891    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26bc4\" (UniqueName: \"kubernetes.io/projected/6e12398d-7b1b-478b-8031-8b0e7617091d-kube-api-access-26bc4\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.888005 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.887915    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmfmc\" (UniqueName: \"kubernetes.io/projected/bc636ec8-6508-407c-8f57-38cf0bcf86a3-kube-api-access-rmfmc\") pod \"node-resolver-wtmzf\" (UID: \"bc636ec8-6508-407c-8f57-38cf0bcf86a3\") " pod="openshift-dns/node-resolver-wtmzf"
Jul 10 00:15:52.888005 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.887947    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e12398d-7b1b-478b-8031-8b0e7617091d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.888005 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.887972    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2rlj\" (UniqueName: \"kubernetes.io/projected/1535944a-df71-496e-955d-7d67d827bec0-kube-api-access-v2rlj\") pod \"network-metrics-daemon-qzdmp\" (UID: \"1535944a-df71-496e-955d-7d67d827bec0\") " pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:15:52.888210 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888032    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e12398d-7b1b-478b-8031-8b0e7617091d-os-release\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.888210 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888091    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e12398d-7b1b-478b-8031-8b0e7617091d-cnibin\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.888210 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888155    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs\") pod \"network-metrics-daemon-qzdmp\" (UID: \"1535944a-df71-496e-955d-7d67d827bec0\") " pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:15:52.888442 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888196    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e12398d-7b1b-478b-8031-8b0e7617091d-cnibin\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.888442 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888214    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bc636ec8-6508-407c-8f57-38cf0bcf86a3-hosts-file\") pod \"node-resolver-wtmzf\" (UID: \"bc636ec8-6508-407c-8f57-38cf0bcf86a3\") " pod="openshift-dns/node-resolver-wtmzf"
Jul 10 00:15:52.888442 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888192    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e12398d-7b1b-478b-8031-8b0e7617091d-os-release\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.888442 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888260    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc636ec8-6508-407c-8f57-38cf0bcf86a3-tmp-dir\") pod \"node-resolver-wtmzf\" (UID: \"bc636ec8-6508-407c-8f57-38cf0bcf86a3\") " pod="openshift-dns/node-resolver-wtmzf"
Jul 10 00:15:52.888442 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888295    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e12398d-7b1b-478b-8031-8b0e7617091d-system-cni-dir\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.888442 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:52.888305    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 00:15:52.888442 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888341    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b9f0bd1-9d79-4814-ae7c-050412fdc4c6-host-slash\") pod \"iptables-alerter-9l7jh\" (UID: \"5b9f0bd1-9d79-4814-ae7c-050412fdc4c6\") " pod="openshift-network-operator/iptables-alerter-9l7jh"
Jul 10 00:15:52.888442 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888368    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bc636ec8-6508-407c-8f57-38cf0bcf86a3-hosts-file\") pod \"node-resolver-wtmzf\" (UID: \"bc636ec8-6508-407c-8f57-38cf0bcf86a3\") " pod="openshift-dns/node-resolver-wtmzf"
Jul 10 00:15:52.888442 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888374    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccctr\" (UniqueName: \"kubernetes.io/projected/5b9f0bd1-9d79-4814-ae7c-050412fdc4c6-kube-api-access-ccctr\") pod \"iptables-alerter-9l7jh\" (UID: \"5b9f0bd1-9d79-4814-ae7c-050412fdc4c6\") " pod="openshift-network-operator/iptables-alerter-9l7jh"
Jul 10 00:15:52.888442 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888378    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e12398d-7b1b-478b-8031-8b0e7617091d-system-cni-dir\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.888442 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:52.888401    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs podName:1535944a-df71-496e-955d-7d67d827bec0 nodeName:}" failed. No retries permitted until 2026-07-10 00:15:53.388379334 +0000 UTC m=+3.091164178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs") pod "network-metrics-daemon-qzdmp" (UID: "1535944a-df71-496e-955d-7d67d827bec0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 00:15:52.888442 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888440    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b9f0bd1-9d79-4814-ae7c-050412fdc4c6-host-slash\") pod \"iptables-alerter-9l7jh\" (UID: \"5b9f0bd1-9d79-4814-ae7c-050412fdc4c6\") " pod="openshift-network-operator/iptables-alerter-9l7jh"
Jul 10 00:15:52.888988 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888469    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e12398d-7b1b-478b-8031-8b0e7617091d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.888988 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888502    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e12398d-7b1b-478b-8031-8b0e7617091d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.888988 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888549    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5b9f0bd1-9d79-4814-ae7c-050412fdc4c6-iptables-alerter-script\") pod \"iptables-alerter-9l7jh\" (UID: \"5b9f0bd1-9d79-4814-ae7c-050412fdc4c6\") " pod="openshift-network-operator/iptables-alerter-9l7jh"
Jul 10 00:15:52.888988 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888622    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e12398d-7b1b-478b-8031-8b0e7617091d-cni-binary-copy\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.888988 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888631    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc636ec8-6508-407c-8f57-38cf0bcf86a3-tmp-dir\") pod \"node-resolver-wtmzf\" (UID: \"bc636ec8-6508-407c-8f57-38cf0bcf86a3\") " pod="openshift-dns/node-resolver-wtmzf"
Jul 10 00:15:52.888988 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888677    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e12398d-7b1b-478b-8031-8b0e7617091d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.888988 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.888941    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e12398d-7b1b-478b-8031-8b0e7617091d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.889274 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.889008    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5b9f0bd1-9d79-4814-ae7c-050412fdc4c6-iptables-alerter-script\") pod \"iptables-alerter-9l7jh\" (UID: \"5b9f0bd1-9d79-4814-ae7c-050412fdc4c6\") " pod="openshift-network-operator/iptables-alerter-9l7jh"
Jul 10 00:15:52.889274 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.889105    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e12398d-7b1b-478b-8031-8b0e7617091d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.899246 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.899221    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmfmc\" (UniqueName: \"kubernetes.io/projected/bc636ec8-6508-407c-8f57-38cf0bcf86a3-kube-api-access-rmfmc\") pod \"node-resolver-wtmzf\" (UID: \"bc636ec8-6508-407c-8f57-38cf0bcf86a3\") " pod="openshift-dns/node-resolver-wtmzf"
Jul 10 00:15:52.899428 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.899406    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26bc4\" (UniqueName: \"kubernetes.io/projected/6e12398d-7b1b-478b-8031-8b0e7617091d-kube-api-access-26bc4\") pod \"multus-additional-cni-plugins-tdzvd\" (UID: \"6e12398d-7b1b-478b-8031-8b0e7617091d\") " pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:52.899510 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.899495    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2rlj\" (UniqueName: \"kubernetes.io/projected/1535944a-df71-496e-955d-7d67d827bec0-kube-api-access-v2rlj\") pod \"network-metrics-daemon-qzdmp\" (UID: \"1535944a-df71-496e-955d-7d67d827bec0\") " pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:15:52.899552 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.899530    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccctr\" (UniqueName: \"kubernetes.io/projected/5b9f0bd1-9d79-4814-ae7c-050412fdc4c6-kube-api-access-ccctr\") pod \"iptables-alerter-9l7jh\" (UID: \"5b9f0bd1-9d79-4814-ae7c-050412fdc4c6\") " pod="openshift-network-operator/iptables-alerter-9l7jh"
Jul 10 00:15:52.973862 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.973778    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8"
Jul 10 00:15:52.976751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.976729    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g88v9"
Jul 10 00:15:52.985022 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:52.984993    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc6182be_be7a_402a_b5b0_584a1e38e26d.slice/crio-a6f2c7149f8d32291e30d766054b6855bc9c9d444eb69e8a6ad65911ec6790e4 WatchSource:0}: Error finding container a6f2c7149f8d32291e30d766054b6855bc9c9d444eb69e8a6ad65911ec6790e4: Status 404 returned error can't find the container with id a6f2c7149f8d32291e30d766054b6855bc9c9d444eb69e8a6ad65911ec6790e4
Jul 10 00:15:52.986884 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:52.986861    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6831fcd_2a29_48d5_aa85_b06e836645c6.slice/crio-4a598747cd15330e7b72eebd13b7de10ab20cd8842c42e2e1f5ba1620bc528bd WatchSource:0}: Error finding container 4a598747cd15330e7b72eebd13b7de10ab20cd8842c42e2e1f5ba1620bc528bd: Status 404 returned error can't find the container with id 4a598747cd15330e7b72eebd13b7de10ab20cd8842c42e2e1f5ba1620bc528bd
Jul 10 00:15:52.987239 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.987225    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-529b2"
Jul 10 00:15:52.992207 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.992190    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:15:52.995188 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:52.995168    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e2cf300_7c42_4aaf_97d0_e85b66d2c2b6.slice/crio-9086878e4eb6d119d800d39590878585e9b720fd5d349aaa27d51895ed6de0bd WatchSource:0}: Error finding container 9086878e4eb6d119d800d39590878585e9b720fd5d349aaa27d51895ed6de0bd: Status 404 returned error can't find the container with id 9086878e4eb6d119d800d39590878585e9b720fd5d349aaa27d51895ed6de0bd
Jul 10 00:15:52.997183 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:52.997164    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cldmd"
Jul 10 00:15:53.001448 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:53.001424    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf75ad69a_e4b3_484a_a93c_468bd16e460c.slice/crio-7f52aad024aa7522267f176a0d53c4b2219e82ffa0a4bed1717130bc96426ad8 WatchSource:0}: Error finding container 7f52aad024aa7522267f176a0d53c4b2219e82ffa0a4bed1717130bc96426ad8: Status 404 returned error can't find the container with id 7f52aad024aa7522267f176a0d53c4b2219e82ffa0a4bed1717130bc96426ad8
Jul 10 00:15:53.003048 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.003032    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gphtp"
Jul 10 00:15:53.008722 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:53.008699    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615763a8_ff81_4ac2_af5a_0b3c7b016222.slice/crio-7280975dc587b8f5cc774f8bbbad2c6d8f0c4ef0548c47413de718eb89f4082d WatchSource:0}: Error finding container 7280975dc587b8f5cc774f8bbbad2c6d8f0c4ef0548c47413de718eb89f4082d: Status 404 returned error can't find the container with id 7280975dc587b8f5cc774f8bbbad2c6d8f0c4ef0548c47413de718eb89f4082d
Jul 10 00:15:53.008808 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.008745    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tdzvd"
Jul 10 00:15:53.011467 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:53.011445    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ba31ebb_c51d_4692_80de_d8a7384b7e2c.slice/crio-0df0fd3b8c12ef3accc7d76626e6518643d381b2b4bcd25659f2968919370264 WatchSource:0}: Error finding container 0df0fd3b8c12ef3accc7d76626e6518643d381b2b4bcd25659f2968919370264: Status 404 returned error can't find the container with id 0df0fd3b8c12ef3accc7d76626e6518643d381b2b4bcd25659f2968919370264
Jul 10 00:15:53.016212 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.016190    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9l7jh"
Jul 10 00:15:53.016654 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:53.016636    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e12398d_7b1b_478b_8031_8b0e7617091d.slice/crio-3245f277a403bc4dc587d53e4da253c37e48f7f9f14869d875fa74f9d9e9398b WatchSource:0}: Error finding container 3245f277a403bc4dc587d53e4da253c37e48f7f9f14869d875fa74f9d9e9398b: Status 404 returned error can't find the container with id 3245f277a403bc4dc587d53e4da253c37e48f7f9f14869d875fa74f9d9e9398b
Jul 10 00:15:53.020462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.020441    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wtmzf"
Jul 10 00:15:53.024618 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:53.024597    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b9f0bd1_9d79_4814_ae7c_050412fdc4c6.slice/crio-041fda1ac5e67f0d5a37be3ef83709a16431924ca719714ea89eab74e147d122 WatchSource:0}: Error finding container 041fda1ac5e67f0d5a37be3ef83709a16431924ca719714ea89eab74e147d122: Status 404 returned error can't find the container with id 041fda1ac5e67f0d5a37be3ef83709a16431924ca719714ea89eab74e147d122
Jul 10 00:15:53.028976 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:15:53.028957    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc636ec8_6508_407c_8f57_38cf0bcf86a3.slice/crio-87dd23b31a593a8955c8d0ba79161d2c04ef861cc69910238d57039e47de7175 WatchSource:0}: Error finding container 87dd23b31a593a8955c8d0ba79161d2c04ef861cc69910238d57039e47de7175: Status 404 returned error can't find the container with id 87dd23b31a593a8955c8d0ba79161d2c04ef861cc69910238d57039e47de7175
Jul 10 00:15:53.292145 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.292048    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret\") pod \"global-pull-secret-syncer-6x94h\" (UID: \"7ce01e87-f4a3-4774-b800-f1dc965f9689\") " pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:15:53.292318 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:53.292223    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 00:15:53.292418 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:53.292322    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret podName:7ce01e87-f4a3-4774-b800-f1dc965f9689 nodeName:}" failed. No retries permitted until 2026-07-10 00:15:54.292290647 +0000 UTC m=+3.995075498 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret") pod "global-pull-secret-syncer-6x94h" (UID: "7ce01e87-f4a3-4774-b800-f1dc965f9689") : object "kube-system"/"original-pull-secret" not registered
Jul 10 00:15:53.393351 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.393303    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs\") pod \"network-metrics-daemon-qzdmp\" (UID: \"1535944a-df71-496e-955d-7d67d827bec0\") " pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:15:53.393351 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.393357    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfdcq\" (UniqueName: \"kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq\") pod \"network-check-target-klmj6\" (UID: \"f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55\") " pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:15:53.393575 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:53.393481    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 00:15:53.393575 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:53.393483    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 00:15:53.393575 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:53.393499    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 00:15:53.393575 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:53.393514    2559 projected.go:196] Error preparing data for projected volume kube-api-access-bfdcq for pod openshift-network-diagnostics/network-check-target-klmj6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 00:15:53.393575 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:53.393573    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs podName:1535944a-df71-496e-955d-7d67d827bec0 nodeName:}" failed. No retries permitted until 2026-07-10 00:15:54.393550591 +0000 UTC m=+4.096335437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs") pod "network-metrics-daemon-qzdmp" (UID: "1535944a-df71-496e-955d-7d67d827bec0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 00:15:53.393889 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:53.393606    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq podName:f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55 nodeName:}" failed. No retries permitted until 2026-07-10 00:15:54.393584318 +0000 UTC m=+4.096369147 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfdcq" (UniqueName: "kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq") pod "network-check-target-klmj6" (UID: "f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 00:15:53.717676 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.717526    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 00:10:53 +0000 UTC" deadline="2028-03-09 07:07:45.550425455 +0000 UTC"
Jul 10 00:15:53.717676 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.717566    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14598h51m51.832864382s"
Jul 10 00:15:53.830870 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.830833    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:15:53.831054 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:53.830981    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:15:53.855999 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.855956    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wtmzf" event={"ID":"bc636ec8-6508-407c-8f57-38cf0bcf86a3","Type":"ContainerStarted","Data":"87dd23b31a593a8955c8d0ba79161d2c04ef861cc69910238d57039e47de7175"}
Jul 10 00:15:53.865251 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.865209    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9l7jh" event={"ID":"5b9f0bd1-9d79-4814-ae7c-050412fdc4c6","Type":"ContainerStarted","Data":"041fda1ac5e67f0d5a37be3ef83709a16431924ca719714ea89eab74e147d122"}
Jul 10 00:15:53.869092 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.869060    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gphtp" event={"ID":"7ba31ebb-c51d-4692-80de-d8a7384b7e2c","Type":"ContainerStarted","Data":"0df0fd3b8c12ef3accc7d76626e6518643d381b2b4bcd25659f2968919370264"}
Jul 10 00:15:53.890326 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.890280    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q" event={"ID":"f75ad69a-e4b3-484a-a93c-468bd16e460c","Type":"ContainerStarted","Data":"7f52aad024aa7522267f176a0d53c4b2219e82ffa0a4bed1717130bc96426ad8"}
Jul 10 00:15:53.901847 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.901719    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8" event={"ID":"fc6182be-be7a-402a-b5b0-584a1e38e26d","Type":"ContainerStarted","Data":"a6f2c7149f8d32291e30d766054b6855bc9c9d444eb69e8a6ad65911ec6790e4"}
Jul 10 00:15:53.915762 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.915721    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdzvd" event={"ID":"6e12398d-7b1b-478b-8031-8b0e7617091d","Type":"ContainerStarted","Data":"3245f277a403bc4dc587d53e4da253c37e48f7f9f14869d875fa74f9d9e9398b"}
Jul 10 00:15:53.925351 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.925311    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cldmd" event={"ID":"615763a8-ff81-4ac2-af5a-0b3c7b016222","Type":"ContainerStarted","Data":"7280975dc587b8f5cc774f8bbbad2c6d8f0c4ef0548c47413de718eb89f4082d"}
Jul 10 00:15:53.929628 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.929595    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-529b2" event={"ID":"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6","Type":"ContainerStarted","Data":"9086878e4eb6d119d800d39590878585e9b720fd5d349aaa27d51895ed6de0bd"}
Jul 10 00:15:53.943142 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:53.943086    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g88v9" event={"ID":"c6831fcd-2a29-48d5-aa85-b06e836645c6","Type":"ContainerStarted","Data":"4a598747cd15330e7b72eebd13b7de10ab20cd8842c42e2e1f5ba1620bc528bd"}
Jul 10 00:15:54.300777 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:54.300723    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret\") pod \"global-pull-secret-syncer-6x94h\" (UID: \"7ce01e87-f4a3-4774-b800-f1dc965f9689\") " pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:15:54.301004 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:54.300961    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 00:15:54.301079 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:54.301035    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret podName:7ce01e87-f4a3-4774-b800-f1dc965f9689 nodeName:}" failed. No retries permitted until 2026-07-10 00:15:56.301014139 +0000 UTC m=+6.003798970 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret") pod "global-pull-secret-syncer-6x94h" (UID: "7ce01e87-f4a3-4774-b800-f1dc965f9689") : object "kube-system"/"original-pull-secret" not registered
Jul 10 00:15:54.402228 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:54.402182    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs\") pod \"network-metrics-daemon-qzdmp\" (UID: \"1535944a-df71-496e-955d-7d67d827bec0\") " pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:15:54.402447 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:54.402243    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfdcq\" (UniqueName: \"kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq\") pod \"network-check-target-klmj6\" (UID: \"f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55\") " pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:15:54.402536 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:54.402507    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 00:15:54.404710 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:54.402592    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs podName:1535944a-df71-496e-955d-7d67d827bec0 nodeName:}" failed. No retries permitted until 2026-07-10 00:15:56.402570179 +0000 UTC m=+6.105355030 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs") pod "network-metrics-daemon-qzdmp" (UID: "1535944a-df71-496e-955d-7d67d827bec0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 00:15:54.404710 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:54.402606    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 00:15:54.404710 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:54.402624    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 00:15:54.404710 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:54.402638    2559 projected.go:196] Error preparing data for projected volume kube-api-access-bfdcq for pod openshift-network-diagnostics/network-check-target-klmj6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 00:15:54.404710 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:54.402683    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq podName:f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55 nodeName:}" failed. No retries permitted until 2026-07-10 00:15:56.40267111 +0000 UTC m=+6.105455938 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfdcq" (UniqueName: "kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq") pod "network-check-target-klmj6" (UID: "f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 00:15:54.718545 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:54.718403    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 00:10:53 +0000 UTC" deadline="2027-12-11 16:02:30.970157023 +0000 UTC"
Jul 10 00:15:54.718545 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:54.718545    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12471h46m36.251617321s"
Jul 10 00:15:54.833829 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:54.833794    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:15:54.833999 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:54.833927    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:15:54.834433 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:54.834378    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:15:54.834549 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:54.834506    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:15:55.830956 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:55.830908    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:15:55.831471 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:55.831054    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:15:56.320297 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:56.320205    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret\") pod \"global-pull-secret-syncer-6x94h\" (UID: \"7ce01e87-f4a3-4774-b800-f1dc965f9689\") " pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:15:56.320493 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:56.320463    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 00:15:56.320558 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:56.320534    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret podName:7ce01e87-f4a3-4774-b800-f1dc965f9689 nodeName:}" failed. No retries permitted until 2026-07-10 00:16:00.320511711 +0000 UTC m=+10.023296542 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret") pod "global-pull-secret-syncer-6x94h" (UID: "7ce01e87-f4a3-4774-b800-f1dc965f9689") : object "kube-system"/"original-pull-secret" not registered
Jul 10 00:15:56.421630 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:56.421588    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs\") pod \"network-metrics-daemon-qzdmp\" (UID: \"1535944a-df71-496e-955d-7d67d827bec0\") " pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:15:56.421828 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:56.421649    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfdcq\" (UniqueName: \"kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq\") pod \"network-check-target-klmj6\" (UID: \"f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55\") " pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:15:56.421828 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:56.421809    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 00:15:56.421828 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:56.421828    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 00:15:56.421996 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:56.421842    2559 projected.go:196] Error preparing data for projected volume kube-api-access-bfdcq for pod openshift-network-diagnostics/network-check-target-klmj6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 00:15:56.421996 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:56.421915    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq podName:f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55 nodeName:}" failed. No retries permitted until 2026-07-10 00:16:00.421894406 +0000 UTC m=+10.124679261 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfdcq" (UniqueName: "kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq") pod "network-check-target-klmj6" (UID: "f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 00:15:56.421996 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:56.421932    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 00:15:56.422173 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:56.422003    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs podName:1535944a-df71-496e-955d-7d67d827bec0 nodeName:}" failed. No retries permitted until 2026-07-10 00:16:00.421983426 +0000 UTC m=+10.124768258 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs") pod "network-metrics-daemon-qzdmp" (UID: "1535944a-df71-496e-955d-7d67d827bec0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 00:15:56.832037 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:56.832003    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:15:56.832037 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:56.832003    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:15:56.832557 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:56.832169    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:15:56.834734 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:56.832657    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:15:57.831186 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:57.831119    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:15:57.831373 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:57.831283    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:15:58.830894 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:58.830818    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:15:58.831507 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:58.830977    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:15:58.831507 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:58.831020    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:15:58.831507 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:58.831125    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:15:59.830684 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:15:59.830647    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:15:59.830897 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:15:59.830791    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:16:00.357258 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:00.357159    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret\") pod \"global-pull-secret-syncer-6x94h\" (UID: \"7ce01e87-f4a3-4774-b800-f1dc965f9689\") " pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:00.357778 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:00.357316    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 00:16:00.357778 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:00.357407    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret podName:7ce01e87-f4a3-4774-b800-f1dc965f9689 nodeName:}" failed. No retries permitted until 2026-07-10 00:16:08.35738596 +0000 UTC m=+18.060170791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret") pod "global-pull-secret-syncer-6x94h" (UID: "7ce01e87-f4a3-4774-b800-f1dc965f9689") : object "kube-system"/"original-pull-secret" not registered
Jul 10 00:16:00.458460 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:00.458416    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs\") pod \"network-metrics-daemon-qzdmp\" (UID: \"1535944a-df71-496e-955d-7d67d827bec0\") " pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:00.458612 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:00.458469    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfdcq\" (UniqueName: \"kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq\") pod \"network-check-target-klmj6\" (UID: \"f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55\") " pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:00.458699 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:00.458604    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 00:16:00.458699 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:00.458676    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs podName:1535944a-df71-496e-955d-7d67d827bec0 nodeName:}" failed. No retries permitted until 2026-07-10 00:16:08.458659106 +0000 UTC m=+18.161443932 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs") pod "network-metrics-daemon-qzdmp" (UID: "1535944a-df71-496e-955d-7d67d827bec0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 00:16:00.458816 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:00.458615    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 00:16:00.458816 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:00.458741    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 00:16:00.458816 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:00.458767    2559 projected.go:196] Error preparing data for projected volume kube-api-access-bfdcq for pod openshift-network-diagnostics/network-check-target-klmj6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 00:16:00.458960 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:00.458832    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq podName:f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55 nodeName:}" failed. No retries permitted until 2026-07-10 00:16:08.458813431 +0000 UTC m=+18.161598272 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfdcq" (UniqueName: "kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq") pod "network-check-target-klmj6" (UID: "f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 00:16:00.831809 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:00.831614    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:00.831809 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:00.831735    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:16:00.832035 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:00.831840    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:00.832035 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:00.831973    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:16:01.830977 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:01.830926    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:01.831656 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:01.831122    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:16:02.830881 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:02.830799    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:02.831047 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:02.830875    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:02.831047 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:02.831017    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:16:02.831456 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:02.831150    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:16:03.830218 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:03.830185    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:03.830390 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:03.830304    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:16:04.830764 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:04.830723    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:04.831166 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:04.830779    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:04.831166 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:04.830870    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:16:04.831166 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:04.831018    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:16:05.830841 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:05.830805    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:05.831222 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:05.830919    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:16:06.830878 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:06.830835    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:06.831350 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:06.830966    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:16:06.831350 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:06.831015    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:06.831350 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:06.831124    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:16:07.831251 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:07.831209    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:07.831783 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:07.831338    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:16:08.420503 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:08.420458    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret\") pod \"global-pull-secret-syncer-6x94h\" (UID: \"7ce01e87-f4a3-4774-b800-f1dc965f9689\") " pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:08.420674 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:08.420610    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 00:16:08.420727 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:08.420688    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret podName:7ce01e87-f4a3-4774-b800-f1dc965f9689 nodeName:}" failed. No retries permitted until 2026-07-10 00:16:24.420664226 +0000 UTC m=+34.123449075 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret") pod "global-pull-secret-syncer-6x94h" (UID: "7ce01e87-f4a3-4774-b800-f1dc965f9689") : object "kube-system"/"original-pull-secret" not registered
Jul 10 00:16:08.521553 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:08.521516    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs\") pod \"network-metrics-daemon-qzdmp\" (UID: \"1535944a-df71-496e-955d-7d67d827bec0\") " pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:08.521553 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:08.521558    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfdcq\" (UniqueName: \"kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq\") pod \"network-check-target-klmj6\" (UID: \"f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55\") " pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:08.521769 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:08.521707    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 00:16:08.521769 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:08.521725    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 00:16:08.521769 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:08.521735    2559 projected.go:196] Error preparing data for projected volume kube-api-access-bfdcq for pod openshift-network-diagnostics/network-check-target-klmj6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 00:16:08.521898 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:08.521706    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 00:16:08.521898 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:08.521794    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq podName:f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55 nodeName:}" failed. No retries permitted until 2026-07-10 00:16:24.521776611 +0000 UTC m=+34.224561438 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfdcq" (UniqueName: "kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq") pod "network-check-target-klmj6" (UID: "f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 00:16:08.521898 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:08.521825    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs podName:1535944a-df71-496e-955d-7d67d827bec0 nodeName:}" failed. No retries permitted until 2026-07-10 00:16:24.521813032 +0000 UTC m=+34.224597876 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs") pod "network-metrics-daemon-qzdmp" (UID: "1535944a-df71-496e-955d-7d67d827bec0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 00:16:08.830531 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:08.830496    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:08.830705 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:08.830628    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:16:08.830705 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:08.830678    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:08.830793 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:08.830776    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:16:09.830460 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:09.830426    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:09.830863 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:09.830546    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:16:10.831805 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:10.831741    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:10.832262 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:10.831863    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:16:10.832262 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:10.831904    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:10.832262 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:10.832021    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:16:11.830660 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:11.830620    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:11.830836 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:11.830741    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:16:12.831325 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:12.831280    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:12.831826 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:12.831429    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:16:12.831826 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:12.831486    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:12.831826 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:12.831601    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:16:13.831037 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:13.830850    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:13.831139 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:13.831120    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:16:13.990733 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:13.990705    2559 generic.go:356] "Generic (PLEG): container finished" podID="43b563d1079fd4e3284b95fba6c82049" containerID="17c090bcf4508a2c69e44e8bf8ca2c0109ca48dc66c443b2798de56fb4a8edfd" exitCode=0
Jul 10 00:16:13.991516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:13.990781    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal" event={"ID":"43b563d1079fd4e3284b95fba6c82049","Type":"ContainerDied","Data":"17c090bcf4508a2c69e44e8bf8ca2c0109ca48dc66c443b2798de56fb4a8edfd"}
Jul 10 00:16:13.992230 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:13.992209    2559 generic.go:356] "Generic (PLEG): container finished" podID="6e12398d-7b1b-478b-8031-8b0e7617091d" containerID="186bb1d9d9b2a2bded100dabdfd347484a7c56614c2076040b31b55751df218d" exitCode=0
Jul 10 00:16:13.992300 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:13.992276    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdzvd" event={"ID":"6e12398d-7b1b-478b-8031-8b0e7617091d","Type":"ContainerDied","Data":"186bb1d9d9b2a2bded100dabdfd347484a7c56614c2076040b31b55751df218d"}
Jul 10 00:16:13.993737 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:13.993713    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cldmd" event={"ID":"615763a8-ff81-4ac2-af5a-0b3c7b016222","Type":"ContainerStarted","Data":"8b83566307e28c881fc649277af7bbc143396ddab5fef230c0e5ec9e3a962eb0"}
Jul 10 00:16:13.995116 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:13.995076    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-529b2" event={"ID":"1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6","Type":"ContainerStarted","Data":"fe20780181f1691312436646d3bbbb6370428d5c23af5fa7eec73ad610fd1e4b"}
Jul 10 00:16:13.996446 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:13.996426    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g88v9" event={"ID":"c6831fcd-2a29-48d5-aa85-b06e836645c6","Type":"ContainerStarted","Data":"1971747ae54d477ec00531269404343efe764453c1de9db4c1a38ba3cdd3af10"}
Jul 10 00:16:13.997730 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:13.997713    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-116.ec2.internal" event={"ID":"92d561d32137a202f7c75fa6c2ab0660","Type":"ContainerStarted","Data":"38f55a7dc0dc8a3257794c1ff5f965716c237ee431324b76568f5c15470ff2ae"}
Jul 10 00:16:13.998890 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:13.998868    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wtmzf" event={"ID":"bc636ec8-6508-407c-8f57-38cf0bcf86a3","Type":"ContainerStarted","Data":"85ac27f8863781b47884844651a9497181ca2427233f4f561f4ce7665e576c51"}
Jul 10 00:16:14.000045 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.000026    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gphtp" event={"ID":"7ba31ebb-c51d-4692-80de-d8a7384b7e2c","Type":"ContainerStarted","Data":"67a06e0769f9664866b72b4ee0e6ed5c676e0d094ea8fe0990715834fdf65649"}
Jul 10 00:16:14.002849 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.002828    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:16:14.003274 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.003189    2559 generic.go:356] "Generic (PLEG): container finished" podID="f75ad69a-e4b3-484a-a93c-468bd16e460c" containerID="26903828017c24ca5c11de14204265507e4391545b514a082b286aa10672d813" exitCode=1
Jul 10 00:16:14.003274 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.003253    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q" event={"ID":"f75ad69a-e4b3-484a-a93c-468bd16e460c","Type":"ContainerStarted","Data":"2b0ec45984a5123d8282df027ba1c862bb13c292108cff72c0f1bdde7d6a474d"}
Jul 10 00:16:14.003401 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.003345    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q" event={"ID":"f75ad69a-e4b3-484a-a93c-468bd16e460c","Type":"ContainerStarted","Data":"8d6f74b2809e0b208034438c1791d5885a7d48142b85166a2a1565ebc89041f4"}
Jul 10 00:16:14.003451 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.003401    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q" event={"ID":"f75ad69a-e4b3-484a-a93c-468bd16e460c","Type":"ContainerStarted","Data":"4659f861dc5b82e2afd79b27e20846489d44e5604a72e0b7f18b07fb181274d6"}
Jul 10 00:16:14.003490 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.003450    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q" event={"ID":"f75ad69a-e4b3-484a-a93c-468bd16e460c","Type":"ContainerDied","Data":"26903828017c24ca5c11de14204265507e4391545b514a082b286aa10672d813"}
Jul 10 00:16:14.003525 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.003490    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q" event={"ID":"f75ad69a-e4b3-484a-a93c-468bd16e460c","Type":"ContainerStarted","Data":"188135903c6d5d7a6eef03aafd73830acd81d22b1bf5ab74fe20200b21fd589b"}
Jul 10 00:16:14.004680 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.004659    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8" event={"ID":"fc6182be-be7a-402a-b5b0-584a1e38e26d","Type":"ContainerStarted","Data":"e4a4fa597585e84c875f8e7e1fd9970a9c3cc02a81a0175d9250aad72d5dfbfb"}
Jul 10 00:16:14.019951 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.019907    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-529b2" podStartSLOduration=2.712480815 podStartE2EDuration="23.019893014s" totalImagesPullingTime="20.307412199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:16:14.019334197 +0000 UTC m=+23.722119066" watchObservedRunningTime="2026-07-10 00:16:14.019893014 +0000 UTC m=+23.722677906"
Jul 10 00:16:14.058084 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.058030    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cldmd" podStartSLOduration=7.304023301 podStartE2EDuration="23.058008516s" totalImagesPullingTime="15.753985215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:16:14.05742086 +0000 UTC m=+23.760205734" watchObservedRunningTime="2026-07-10 00:16:14.058008516 +0000 UTC m=+23.760793367"
Jul 10 00:16:14.076160 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.076104    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gphtp" podStartSLOduration=2.8386437 podStartE2EDuration="23.076084011s" totalImagesPullingTime="20.237440311s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:16:14.075870328 +0000 UTC m=+23.778655187" watchObservedRunningTime="2026-07-10 00:16:14.076084011 +0000 UTC m=+23.778868860"
Jul 10 00:16:14.114222 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.114179    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-116.ec2.internal" podStartSLOduration=22.114163055 podStartE2EDuration="22.114163055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:15:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:16:14.113898807 +0000 UTC m=+23.816683655" watchObservedRunningTime="2026-07-10 00:16:14.114163055 +0000 UTC m=+23.816947902"
Jul 10 00:16:14.131034 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.130984    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-g88v9" podStartSLOduration=2.8371122570000002 podStartE2EDuration="23.130970067s" totalImagesPullingTime="20.29385781s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:16:14.130853042 +0000 UTC m=+23.833637889" watchObservedRunningTime="2026-07-10 00:16:14.130970067 +0000 UTC m=+23.833754938"
Jul 10 00:16:14.146060 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.145998    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wtmzf" podStartSLOduration=7.411643626 podStartE2EDuration="23.145973564s" totalImagesPullingTime="15.734329938s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:16:14.1459658 +0000 UTC m=+23.848750665" watchObservedRunningTime="2026-07-10 00:16:14.145973564 +0000 UTC m=+23.848758407"
Jul 10 00:16:14.167793 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.167758    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:14.223352 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.223304    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:14.276421 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.276308    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:14.830423 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.830395    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:14.830423 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:14.830410    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:14.830613 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:14.830526    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:16:14.830697 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:14.830666    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:16:15.009036 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.008995    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal" event={"ID":"43b563d1079fd4e3284b95fba6c82049","Type":"ContainerStarted","Data":"01e68a8c33b8cca3305cb3f10f1a93f5266a30307947c2a64d1402e4e65251f6"}
Jul 10 00:16:15.010772 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.010738    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9l7jh" event={"ID":"5b9f0bd1-9d79-4814-ae7c-050412fdc4c6","Type":"ContainerStarted","Data":"8ad550d97c3e8a0edee3e07fe295d5dafec67b1042018866bb64cf13441f1921"}
Jul 10 00:16:15.014186 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.014165    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:16:15.014680 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.014657    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q" event={"ID":"f75ad69a-e4b3-484a-a93c-468bd16e460c","Type":"ContainerStarted","Data":"99db7ff68e38ece39b17eae43c70883547b4aa9b6c6e418a4a42d4d1e9a10135"}
Jul 10 00:16:15.036913 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.036872    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-116.ec2.internal" podStartSLOduration=23.036857084 podStartE2EDuration="23.036857084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:15:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:16:15.023989176 +0000 UTC m=+24.726774052" watchObservedRunningTime="2026-07-10 00:16:15.036857084 +0000 UTC m=+24.739641932"
Jul 10 00:16:15.037027 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.036999    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9l7jh" podStartSLOduration=3.8132604949999998 podStartE2EDuration="24.036991496s" totalImagesPullingTime="20.223731001s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:16:15.036991532 +0000 UTC m=+24.739776378" watchObservedRunningTime="2026-07-10 00:16:15.036991496 +0000 UTC m=+24.739776346"
Jul 10 00:16:15.153608 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.153573    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:15.154030 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.153909    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 10 00:16:15.253668 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.253637    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:15.359110 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.359031    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cldmd"
Jul 10 00:16:15.359710 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.359687    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cldmd"
Jul 10 00:16:15.423597 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.423567    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:15.472878 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.472842    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:15.750690 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.750601    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T00:16:15.15392823Z","UUID":"41c06aa5-7ec7-46c9-8188-5091a09bba34","Handler":null,"Name":"","Endpoint":""}
Jul 10 00:16:15.752237 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.752215    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 10 00:16:15.752378 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.752255    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 10 00:16:15.830606 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:15.830568    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:15.830769 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:15.830702    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:16:16.018605 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:16.018513    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8" event={"ID":"fc6182be-be7a-402a-b5b0-584a1e38e26d","Type":"ContainerStarted","Data":"450994c40ef81a06297da5b5e3e351839cf49889148299699fe4636d02c0cc8c"}
Jul 10 00:16:16.019050 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:16.019027    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cldmd"
Jul 10 00:16:16.019375 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:16.019359    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cldmd"
Jul 10 00:16:16.084944 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:16.084911    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:16.830411 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:16.830386    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:16.830567 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:16.830396    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:16.830567 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:16.830501    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:16:16.830737 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:16.830592    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:16:17.024662 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:17.024466    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:16:17.025111 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:17.025083    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q" event={"ID":"f75ad69a-e4b3-484a-a93c-468bd16e460c","Type":"ContainerStarted","Data":"e3bcdb96db0027a704e034c00a492a893cbfb8214ecebcf965fe138aa650907e"}
Jul 10 00:16:17.027423 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:17.027387    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8" event={"ID":"fc6182be-be7a-402a-b5b0-584a1e38e26d","Type":"ContainerStarted","Data":"820f5f30014e48c163a479879ce33c34d7b5a755cdbc664876b7f4da4f725540"}
Jul 10 00:16:17.826969 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:17.826932    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:17.831053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:17.831033    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:17.831174 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:17.831152    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:16:17.954176 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:17.954145    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:18.830596 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:18.830562    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:18.831171 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:18.830607    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:18.831171 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:18.830728    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:16:18.831171 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:18.830846    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:16:19.831148 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:19.831114    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:19.831549 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:19.831218    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:16:20.038979 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:20.038962    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:16:20.039290 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:20.039266    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q" event={"ID":"f75ad69a-e4b3-484a-a93c-468bd16e460c","Type":"ContainerStarted","Data":"f8ea9a79c6b374403e3f36cf8c79ac220dac2fa190e0e378f822ee0df02cded0"}
Jul 10 00:16:20.039583 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:20.039562    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:16:20.039712 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:20.039694    2559 scope.go:117] "RemoveContainer" containerID="26903828017c24ca5c11de14204265507e4391545b514a082b286aa10672d813"
Jul 10 00:16:20.055721 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:20.055696    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:16:20.066954 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:20.066779    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfxc8" podStartSLOduration=5.795152966 podStartE2EDuration="29.066752712s" totalImagesPullingTime="23.271599746s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:15:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:16:17.045340125 +0000 UTC m=+26.748124990" watchObservedRunningTime="2026-07-10 00:16:20.066752712 +0000 UTC m=+29.769537553"
Jul 10 00:16:20.831735 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:20.831704    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:20.832365 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:20.831774    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:20.832365 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:20.831788    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:16:20.832365 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:20.831860    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:16:21.042688 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:21.042605    2559 generic.go:356] "Generic (PLEG): container finished" podID="6e12398d-7b1b-478b-8031-8b0e7617091d" containerID="74c37abe1561aa956c294e075c9dde079e774b9d9158196e5ca5d3557f4346da" exitCode=0
Jul 10 00:16:21.042908 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:21.042687    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdzvd" event={"ID":"6e12398d-7b1b-478b-8031-8b0e7617091d","Type":"ContainerDied","Data":"74c37abe1561aa956c294e075c9dde079e774b9d9158196e5ca5d3557f4346da"}
Jul 10 00:16:21.046498 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:21.046481    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:16:21.046828 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:21.046805    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q" event={"ID":"f75ad69a-e4b3-484a-a93c-468bd16e460c","Type":"ContainerStarted","Data":"73d670e9af8cf0a1f7804aeb08b455aa570dea2a22d03922fe3aac8c7dc43ef3"}
Jul 10 00:16:21.047029 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:21.047005    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:16:21.047152 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:21.047106    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:16:21.063225 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:21.063200    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:16:21.087779 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:21.087686    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q" podStartSLOduration=9.751598448 podStartE2EDuration="30.087670604s" totalImagesPullingTime="20.336072156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:16:21.086228553 +0000 UTC m=+30.789013400" watchObservedRunningTime="2026-07-10 00:16:21.087670604 +0000 UTC m=+30.790455451"
Jul 10 00:16:21.605455 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:21.605419    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:21.653967 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:21.653932    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:21.830224 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:21.830199    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:21.830382 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:21.830299    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:16:22.200413 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:22.200384    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-klmj6"]
Jul 10 00:16:22.200827 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:22.200533    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:22.200827 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:22.200632    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:16:22.202790 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:22.202752    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6x94h"]
Jul 10 00:16:22.202921 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:22.202842    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:22.202980 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:22.202935    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:16:22.203526 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:22.203509    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qzdmp"]
Jul 10 00:16:22.203600 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:22.203594    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:22.203693 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:22.203676    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:16:23.831141 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:23.830943    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:23.831569 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:23.830977    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:23.831569 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:23.831232    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:16:23.831569 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:23.830978    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:23.831569 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:23.831300    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:16:23.831569 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:23.831405    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:16:24.055942 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:24.055910    2559 generic.go:356] "Generic (PLEG): container finished" podID="6e12398d-7b1b-478b-8031-8b0e7617091d" containerID="41884c372e5303f7a3ea3f5d793db877690ffaccaabbaad439115c43d983575d" exitCode=0
Jul 10 00:16:24.056071 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:24.055952    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdzvd" event={"ID":"6e12398d-7b1b-478b-8031-8b0e7617091d","Type":"ContainerDied","Data":"41884c372e5303f7a3ea3f5d793db877690ffaccaabbaad439115c43d983575d"}
Jul 10 00:16:24.444918 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:24.444887    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret\") pod \"global-pull-secret-syncer-6x94h\" (UID: \"7ce01e87-f4a3-4774-b800-f1dc965f9689\") " pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:24.445082 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:24.444997    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 00:16:24.445082 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:24.445055    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret podName:7ce01e87-f4a3-4774-b800-f1dc965f9689 nodeName:}" failed. No retries permitted until 2026-07-10 00:16:56.445039441 +0000 UTC m=+66.147824267 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret") pod "global-pull-secret-syncer-6x94h" (UID: "7ce01e87-f4a3-4774-b800-f1dc965f9689") : object "kube-system"/"original-pull-secret" not registered
Jul 10 00:16:24.466432 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:24.466400    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:24.518684 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:24.518652    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:24.545718 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:24.545694    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfdcq\" (UniqueName: \"kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq\") pod \"network-check-target-klmj6\" (UID: \"f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55\") " pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:24.545822 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:24.545748    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs\") pod \"network-metrics-daemon-qzdmp\" (UID: \"1535944a-df71-496e-955d-7d67d827bec0\") " pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:24.545859 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:24.545828    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 00:16:24.545859 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:24.545845    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 00:16:24.545921 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:24.545862    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 00:16:24.545921 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:24.545871    2559 projected.go:196] Error preparing data for projected volume kube-api-access-bfdcq for pod openshift-network-diagnostics/network-check-target-klmj6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 00:16:24.545921 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:24.545873    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs podName:1535944a-df71-496e-955d-7d67d827bec0 nodeName:}" failed. No retries permitted until 2026-07-10 00:16:56.545861244 +0000 UTC m=+66.248646069 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs") pod "network-metrics-daemon-qzdmp" (UID: "1535944a-df71-496e-955d-7d67d827bec0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 00:16:24.545921 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:24.545915    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq podName:f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55 nodeName:}" failed. No retries permitted until 2026-07-10 00:16:56.545904269 +0000 UTC m=+66.248689096 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bfdcq" (UniqueName: "kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq") pod "network-check-target-klmj6" (UID: "f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 00:16:25.059800 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:25.059721    2559 generic.go:356] "Generic (PLEG): container finished" podID="6e12398d-7b1b-478b-8031-8b0e7617091d" containerID="6626720813e1f223a85504760e293d9bd939745bc9195140da1637e940d85674" exitCode=0
Jul 10 00:16:25.059800 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:25.059781    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdzvd" event={"ID":"6e12398d-7b1b-478b-8031-8b0e7617091d","Type":"ContainerDied","Data":"6626720813e1f223a85504760e293d9bd939745bc9195140da1637e940d85674"}
Jul 10 00:16:25.830509 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:25.830469    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:25.830709 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:25.830470    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:25.830709 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:25.830470    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:25.830709 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:25.830629    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:16:25.830866 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:25.830727    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6x94h" podUID="7ce01e87-f4a3-4774-b800-f1dc965f9689"
Jul 10 00:16:25.830866 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:25.830801    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-klmj6" podUID="f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55"
Jul 10 00:16:27.088645 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.088581    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-116.ec2.internal" event="NodeReady"
Jul 10 00:16:27.088977 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.088686    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 10 00:16:27.126641 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.126602    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-j8x4j"]
Jul 10 00:16:27.128415 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.128392    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:27.130063 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.130042    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vtm7t"]
Jul 10 00:16:27.132074 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.132054    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 10 00:16:27.132192 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.132057    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vtm7t"
Jul 10 00:16:27.132252 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.132201    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-crl8x\""
Jul 10 00:16:27.132935 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.132917    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 10 00:16:27.134445 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.134424    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vlbng\""
Jul 10 00:16:27.134531 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.134476    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 10 00:16:27.134744 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.134731    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 10 00:16:27.135155 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.135134    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 10 00:16:27.139936 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.139908    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j8x4j"]
Jul 10 00:16:27.143090 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.143070    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vtm7t"]
Jul 10 00:16:27.230241 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.230202    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:27.268252 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.268226    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8zpd\" (UniqueName: \"kubernetes.io/projected/6e7a4774-431a-40c0-8c44-f3f929be8e0f-kube-api-access-n8zpd\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:27.268252 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.268264    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d2b0c9d-41f9-45be-a60c-52fa7cb19f15-cert\") pod \"ingress-canary-vtm7t\" (UID: \"1d2b0c9d-41f9-45be-a60c-52fa7cb19f15\") " pod="openshift-ingress-canary/ingress-canary-vtm7t"
Jul 10 00:16:27.268514 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.268369    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:27.268514 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.268402    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6e7a4774-431a-40c0-8c44-f3f929be8e0f-config-volume\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:27.268514 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.268486    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4mtc\" (UniqueName: \"kubernetes.io/projected/1d2b0c9d-41f9-45be-a60c-52fa7cb19f15-kube-api-access-r4mtc\") pod \"ingress-canary-vtm7t\" (UID: \"1d2b0c9d-41f9-45be-a60c-52fa7cb19f15\") " pod="openshift-ingress-canary/ingress-canary-vtm7t"
Jul 10 00:16:27.268632 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.268519    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e7a4774-431a-40c0-8c44-f3f929be8e0f-tmp-dir\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:27.280968 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.280939    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:27.369907 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.369827    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:27.369907 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.369875    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6e7a4774-431a-40c0-8c44-f3f929be8e0f-config-volume\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:27.370093 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.369935    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4mtc\" (UniqueName: \"kubernetes.io/projected/1d2b0c9d-41f9-45be-a60c-52fa7cb19f15-kube-api-access-r4mtc\") pod \"ingress-canary-vtm7t\" (UID: \"1d2b0c9d-41f9-45be-a60c-52fa7cb19f15\") " pod="openshift-ingress-canary/ingress-canary-vtm7t"
Jul 10 00:16:27.370093 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.369960    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e7a4774-431a-40c0-8c44-f3f929be8e0f-tmp-dir\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:27.370093 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:27.369983    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 00:16:27.370093 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.369995    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8zpd\" (UniqueName: \"kubernetes.io/projected/6e7a4774-431a-40c0-8c44-f3f929be8e0f-kube-api-access-n8zpd\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:27.370093 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.370028    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d2b0c9d-41f9-45be-a60c-52fa7cb19f15-cert\") pod \"ingress-canary-vtm7t\" (UID: \"1d2b0c9d-41f9-45be-a60c-52fa7cb19f15\") " pod="openshift-ingress-canary/ingress-canary-vtm7t"
Jul 10 00:16:27.370093 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:27.370055    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls podName:6e7a4774-431a-40c0-8c44-f3f929be8e0f nodeName:}" failed. No retries permitted until 2026-07-10 00:16:27.870031958 +0000 UTC m=+37.572816785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls") pod "dns-default-j8x4j" (UID: "6e7a4774-431a-40c0-8c44-f3f929be8e0f") : secret "dns-default-metrics-tls" not found
Jul 10 00:16:27.370518 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.370469    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e7a4774-431a-40c0-8c44-f3f929be8e0f-tmp-dir\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:27.370643 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.370530    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6e7a4774-431a-40c0-8c44-f3f929be8e0f-config-volume\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:27.375537 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.375287    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d2b0c9d-41f9-45be-a60c-52fa7cb19f15-cert\") pod \"ingress-canary-vtm7t\" (UID: \"1d2b0c9d-41f9-45be-a60c-52fa7cb19f15\") " pod="openshift-ingress-canary/ingress-canary-vtm7t"
Jul 10 00:16:27.379289 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.379265    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4mtc\" (UniqueName: \"kubernetes.io/projected/1d2b0c9d-41f9-45be-a60c-52fa7cb19f15-kube-api-access-r4mtc\") pod \"ingress-canary-vtm7t\" (UID: \"1d2b0c9d-41f9-45be-a60c-52fa7cb19f15\") " pod="openshift-ingress-canary/ingress-canary-vtm7t"
Jul 10 00:16:27.381329 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.381291    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8zpd\" (UniqueName: \"kubernetes.io/projected/6e7a4774-431a-40c0-8c44-f3f929be8e0f-kube-api-access-n8zpd\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:27.445384 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.445341    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vtm7t"
Jul 10 00:16:27.579822 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.579576    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vtm7t"]
Jul 10 00:16:27.583357 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:16:27.583306    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d2b0c9d_41f9_45be_a60c_52fa7cb19f15.slice/crio-8f2716ae2ae55f03e9ed576d35725f9bef813e3657c4fedddc58c4f817de8c58 WatchSource:0}: Error finding container 8f2716ae2ae55f03e9ed576d35725f9bef813e3657c4fedddc58c4f817de8c58: Status 404 returned error can't find the container with id 8f2716ae2ae55f03e9ed576d35725f9bef813e3657c4fedddc58c4f817de8c58
Jul 10 00:16:27.830248 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.830216    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:27.830420 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.830260    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:27.830420 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.830260    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:27.834005 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.833977    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 00:16:27.834005 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.833988    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qmtcf\""
Jul 10 00:16:27.834175 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.834016    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 00:16:27.834175 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.834062    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 00:16:27.834175 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.833990    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 00:16:27.834175 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.834104    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qn6qf\""
Jul 10 00:16:27.875236 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:27.875213    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:27.875388 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:27.875370    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 00:16:27.875445 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:27.875435    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls podName:6e7a4774-431a-40c0-8c44-f3f929be8e0f nodeName:}" failed. No retries permitted until 2026-07-10 00:16:28.875417105 +0000 UTC m=+38.578201952 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls") pod "dns-default-j8x4j" (UID: "6e7a4774-431a-40c0-8c44-f3f929be8e0f") : secret "dns-default-metrics-tls" not found
Jul 10 00:16:28.068229 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:28.068148    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vtm7t" event={"ID":"1d2b0c9d-41f9-45be-a60c-52fa7cb19f15","Type":"ContainerStarted","Data":"8f2716ae2ae55f03e9ed576d35725f9bef813e3657c4fedddc58c4f817de8c58"}
Jul 10 00:16:28.885068 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:28.885003    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:28.885790 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:28.885145    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 00:16:28.885790 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:28.885230    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls podName:6e7a4774-431a-40c0-8c44-f3f929be8e0f nodeName:}" failed. No retries permitted until 2026-07-10 00:16:30.885209931 +0000 UTC m=+40.587994772 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls") pod "dns-default-j8x4j" (UID: "6e7a4774-431a-40c0-8c44-f3f929be8e0f") : secret "dns-default-metrics-tls" not found
Jul 10 00:16:30.431827 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:30.431793    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:30.486210 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:30.486178    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:30.898599 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:30.898565    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:30.898731 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:30.898716    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 00:16:30.898795 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:30.898785    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls podName:6e7a4774-431a-40c0-8c44-f3f929be8e0f nodeName:}" failed. No retries permitted until 2026-07-10 00:16:34.898767112 +0000 UTC m=+44.601551946 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls") pod "dns-default-j8x4j" (UID: "6e7a4774-431a-40c0-8c44-f3f929be8e0f") : secret "dns-default-metrics-tls" not found
Jul 10 00:16:31.079531 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:31.079493    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vtm7t" event={"ID":"1d2b0c9d-41f9-45be-a60c-52fa7cb19f15","Type":"ContainerStarted","Data":"208126a9b5e808bbb7af3939ebb3c1fda337e5a0d7588bfbeea0ac6f5480df31"}
Jul 10 00:16:31.082629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:31.082588    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdzvd" event={"ID":"6e12398d-7b1b-478b-8031-8b0e7617091d","Type":"ContainerStarted","Data":"3e57881e2dc1dc3d41778794a5ee9bbce835ac63f7981f48b0cf6c1c133bb08b"}
Jul 10 00:16:31.096140 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:31.096093    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vtm7t" podStartSLOduration=0.800958883 podStartE2EDuration="4.096080118s" totalImagesPullingTime="3.295121235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:16:31.094759506 +0000 UTC m=+40.797544375" watchObservedRunningTime="2026-07-10 00:16:31.096080118 +0000 UTC m=+40.798864945"
Jul 10 00:16:31.826120 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:31.825829    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:31.880363 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:31.880274    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:32.087280 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:32.087251    2559 generic.go:356] "Generic (PLEG): container finished" podID="6e12398d-7b1b-478b-8031-8b0e7617091d" containerID="3e57881e2dc1dc3d41778794a5ee9bbce835ac63f7981f48b0cf6c1c133bb08b" exitCode=0
Jul 10 00:16:32.087442 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:32.087341    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdzvd" event={"ID":"6e12398d-7b1b-478b-8031-8b0e7617091d","Type":"ContainerDied","Data":"3e57881e2dc1dc3d41778794a5ee9bbce835ac63f7981f48b0cf6c1c133bb08b"}
Jul 10 00:16:33.092403 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:33.092368    2559 generic.go:356] "Generic (PLEG): container finished" podID="6e12398d-7b1b-478b-8031-8b0e7617091d" containerID="4b2279b70e3fd56071e04a990743c2807d0ccb5e2f735868c458493a017cf95d" exitCode=0
Jul 10 00:16:33.092766 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:33.092411    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdzvd" event={"ID":"6e12398d-7b1b-478b-8031-8b0e7617091d","Type":"ContainerDied","Data":"4b2279b70e3fd56071e04a990743c2807d0ccb5e2f735868c458493a017cf95d"}
Jul 10 00:16:34.097883 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:34.097850    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tdzvd" event={"ID":"6e12398d-7b1b-478b-8031-8b0e7617091d","Type":"ContainerStarted","Data":"e0661b8219c5ebc194e59c98388c15b3033c23774e7e7565c0db35dedeb615b5"}
Jul 10 00:16:34.120136 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:34.120075    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tdzvd" podStartSLOduration=8.108362322 podStartE2EDuration="43.120056234s" totalImagesPullingTime="35.011693912s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:15:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:16:34.11809918 +0000 UTC m=+43.820884028" watchObservedRunningTime="2026-07-10 00:16:34.120056234 +0000 UTC m=+43.822841082"
Jul 10 00:16:34.237171 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:34.237143    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:34.292060 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:34.292041    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:34.930409 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:34.930370    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:34.930581 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:34.930521    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 00:16:34.930626 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:34.930592    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls podName:6e7a4774-431a-40c0-8c44-f3f929be8e0f nodeName:}" failed. No retries permitted until 2026-07-10 00:16:42.930572888 +0000 UTC m=+52.633357715 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls") pod "dns-default-j8x4j" (UID: "6e7a4774-431a-40c0-8c44-f3f929be8e0f") : secret "dns-default-metrics-tls" not found
Jul 10 00:16:39.098635 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:39.098593    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:39.154174 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:39.154141    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:42.983617 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:42.983581    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:42.984090 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:42.983760    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 00:16:42.984090 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:42.983839    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls podName:6e7a4774-431a-40c0-8c44-f3f929be8e0f nodeName:}" failed. No retries permitted until 2026-07-10 00:16:58.98382052 +0000 UTC m=+68.686605347 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls") pod "dns-default-j8x4j" (UID: "6e7a4774-431a-40c0-8c44-f3f929be8e0f") : secret "dns-default-metrics-tls" not found
Jul 10 00:16:43.447886 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:43.447836    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:43.498990 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:43.498958    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:53.068855 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:53.068825    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2s7q"
Jul 10 00:16:53.170722 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:53.170678    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:53.219509 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:53.219476    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:53.273924 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:53.273897    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:16:56.477040 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:56.476996    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret\") pod \"global-pull-secret-syncer-6x94h\" (UID: \"7ce01e87-f4a3-4774-b800-f1dc965f9689\") " pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:56.479708 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:56.479689    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 00:16:56.491319 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:56.491297    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce01e87-f4a3-4774-b800-f1dc965f9689-original-pull-secret\") pod \"global-pull-secret-syncer-6x94h\" (UID: \"7ce01e87-f4a3-4774-b800-f1dc965f9689\") " pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:56.577842 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:56.577803    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs\") pod \"network-metrics-daemon-qzdmp\" (UID: \"1535944a-df71-496e-955d-7d67d827bec0\") " pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:16:56.577842 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:56.577848    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfdcq\" (UniqueName: \"kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq\") pod \"network-check-target-klmj6\" (UID: \"f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55\") " pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:56.580367 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:56.580347    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 00:16:56.580413 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:56.580347    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 00:16:56.588554 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:56.588525    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 10 00:16:56.588616 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:56.588593    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs podName:1535944a-df71-496e-955d-7d67d827bec0 nodeName:}" failed. No retries permitted until 2026-07-10 00:18:00.588577972 +0000 UTC m=+130.291362811 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs") pod "network-metrics-daemon-qzdmp" (UID: "1535944a-df71-496e-955d-7d67d827bec0") : secret "metrics-daemon-secret" not found
Jul 10 00:16:56.590747 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:56.590729    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 00:16:56.600954 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:56.600925    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfdcq\" (UniqueName: \"kubernetes.io/projected/f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55-kube-api-access-bfdcq\") pod \"network-check-target-klmj6\" (UID: \"f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55\") " pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:56.648478 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:56.648456    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qmtcf\""
Jul 10 00:16:56.650409 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:56.650394    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6x94h"
Jul 10 00:16:56.656817 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:56.656800    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:16:56.774521 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:56.774455    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6x94h"]
Jul 10 00:16:56.778223 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:16:56.778198    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce01e87_f4a3_4774_b800_f1dc965f9689.slice/crio-792e30b725bcc7c6a90b98f072fed4e805c8a1d7695d3e114e9d8d58d152bf56 WatchSource:0}: Error finding container 792e30b725bcc7c6a90b98f072fed4e805c8a1d7695d3e114e9d8d58d152bf56: Status 404 returned error can't find the container with id 792e30b725bcc7c6a90b98f072fed4e805c8a1d7695d3e114e9d8d58d152bf56
Jul 10 00:16:56.791561 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:56.791532    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-klmj6"]
Jul 10 00:16:56.794185 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:16:56.794146    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4e4dd44_71fd_4ad8_ad9c_03a64a65ee55.slice/crio-0936f61604238cb747b58c3292c511231096cdf1dc76f77619fa0cd36ba6b366 WatchSource:0}: Error finding container 0936f61604238cb747b58c3292c511231096cdf1dc76f77619fa0cd36ba6b366: Status 404 returned error can't find the container with id 0936f61604238cb747b58c3292c511231096cdf1dc76f77619fa0cd36ba6b366
Jul 10 00:16:57.150304 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:57.150216    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-klmj6" event={"ID":"f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55","Type":"ContainerStarted","Data":"0936f61604238cb747b58c3292c511231096cdf1dc76f77619fa0cd36ba6b366"}
Jul 10 00:16:57.151151 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:57.151129    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6x94h" event={"ID":"7ce01e87-f4a3-4774-b800-f1dc965f9689","Type":"ContainerStarted","Data":"792e30b725bcc7c6a90b98f072fed4e805c8a1d7695d3e114e9d8d58d152bf56"}
Jul 10 00:16:58.994520 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:16:58.994483    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:16:58.994912 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:58.994648    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 00:16:58.994912 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:16:58.994720    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls podName:6e7a4774-431a-40c0-8c44-f3f929be8e0f nodeName:}" failed. No retries permitted until 2026-07-10 00:17:30.994698328 +0000 UTC m=+100.697483158 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls") pod "dns-default-j8x4j" (UID: "6e7a4774-431a-40c0-8c44-f3f929be8e0f") : secret "dns-default-metrics-tls" not found
Jul 10 00:17:02.166332 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:02.166261    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6x94h" event={"ID":"7ce01e87-f4a3-4774-b800-f1dc965f9689","Type":"ContainerStarted","Data":"97bba4ce42fd3e9e0388dfa0faca16c56ece4360fc20d8c9db2794e7e2f66a27"}
Jul 10 00:17:02.167727 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:02.167705    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-klmj6" event={"ID":"f4e4dd44-71fd-4ad8-ad9c-03a64a65ee55","Type":"ContainerStarted","Data":"dda67c3198c0b12ed0a4869887d4ca39d1ded67908632dd0d6e17df7596ff5a6"}
Jul 10 00:17:02.167857 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:02.167841    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:17:02.180274 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:02.180235    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6x94h" podStartSLOduration=65.966554819 podStartE2EDuration="1m11.180222074s" totalImagesPullingTime="5.213667255s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:17:02.179619936 +0000 UTC m=+71.882404809" watchObservedRunningTime="2026-07-10 00:17:02.180222074 +0000 UTC m=+71.883006922"
Jul 10 00:17:02.193669 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:02.193610    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-klmj6" podStartSLOduration=65.998683046 podStartE2EDuration="1m11.193591484s" totalImagesPullingTime="5.194908438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:17:02.193385203 +0000 UTC m=+71.896170067" watchObservedRunningTime="2026-07-10 00:17:02.193591484 +0000 UTC m=+71.896376312"
Jul 10 00:17:20.584999 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:20.584964    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc"]
Jul 10 00:17:20.587632 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:20.587613    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc"
Jul 10 00:17:20.591823 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:20.591800    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 10 00:17:20.591945 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:20.591908    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 10 00:17:20.592010 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:20.591955    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 10 00:17:20.592074 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:20.592023    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 10 00:17:20.592126 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:20.592074    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-v2c4j\""
Jul 10 00:17:20.597609 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:20.597588    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc"]
Jul 10 00:17:20.638135 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:20.638112    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88345d5f-b0a0-4fb8-b294-6e47d47424b9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc\" (UID: \"88345d5f-b0a0-4fb8-b294-6e47d47424b9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc"
Jul 10 00:17:20.638255 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:20.638144    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57bhs\" (UniqueName: \"kubernetes.io/projected/88345d5f-b0a0-4fb8-b294-6e47d47424b9-kube-api-access-57bhs\") pod \"managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc\" (UID: \"88345d5f-b0a0-4fb8-b294-6e47d47424b9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc"
Jul 10 00:17:20.739078 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:20.739021    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88345d5f-b0a0-4fb8-b294-6e47d47424b9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc\" (UID: \"88345d5f-b0a0-4fb8-b294-6e47d47424b9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc"
Jul 10 00:17:20.739078 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:20.739081    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57bhs\" (UniqueName: \"kubernetes.io/projected/88345d5f-b0a0-4fb8-b294-6e47d47424b9-kube-api-access-57bhs\") pod \"managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc\" (UID: \"88345d5f-b0a0-4fb8-b294-6e47d47424b9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc"
Jul 10 00:17:20.741430 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:20.741406    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88345d5f-b0a0-4fb8-b294-6e47d47424b9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc\" (UID: \"88345d5f-b0a0-4fb8-b294-6e47d47424b9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc"
Jul 10 00:17:20.746798 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:20.746774    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57bhs\" (UniqueName: \"kubernetes.io/projected/88345d5f-b0a0-4fb8-b294-6e47d47424b9-kube-api-access-57bhs\") pod \"managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc\" (UID: \"88345d5f-b0a0-4fb8-b294-6e47d47424b9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc"
Jul 10 00:17:20.907850 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:20.907765    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc"
Jul 10 00:17:21.021570 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:21.021533    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc"]
Jul 10 00:17:21.024647 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:17:21.024623    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88345d5f_b0a0_4fb8_b294_6e47d47424b9.slice/crio-59ca5f4ba85cfa273e700caa8bda59a148a5cca05cbf0028389f497fd59369ea WatchSource:0}: Error finding container 59ca5f4ba85cfa273e700caa8bda59a148a5cca05cbf0028389f497fd59369ea: Status 404 returned error can't find the container with id 59ca5f4ba85cfa273e700caa8bda59a148a5cca05cbf0028389f497fd59369ea
Jul 10 00:17:21.214912 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:21.214880    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc" event={"ID":"88345d5f-b0a0-4fb8-b294-6e47d47424b9","Type":"ContainerStarted","Data":"59ca5f4ba85cfa273e700caa8bda59a148a5cca05cbf0028389f497fd59369ea"}
Jul 10 00:17:24.224656 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:24.224619    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc" event={"ID":"88345d5f-b0a0-4fb8-b294-6e47d47424b9","Type":"ContainerStarted","Data":"091edc4b4792e79840f71a5e92437ca2f70765cde679494dbdb18249111601ae"}
Jul 10 00:17:24.240253 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:24.240208    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc" podStartSLOduration=1.416210561 podStartE2EDuration="4.240193646s" totalImagesPullingTime="2.823983085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:17:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:17:24.238566464 +0000 UTC m=+93.941351313" watchObservedRunningTime="2026-07-10 00:17:24.240193646 +0000 UTC m=+93.942978494"
Jul 10 00:17:31.004003 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:31.003962    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:17:31.004418 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:17:31.004129    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 00:17:31.004418 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:17:31.004210    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls podName:6e7a4774-431a-40c0-8c44-f3f929be8e0f nodeName:}" failed. No retries permitted until 2026-07-10 00:18:35.004190901 +0000 UTC m=+164.706975727 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls") pod "dns-default-j8x4j" (UID: "6e7a4774-431a-40c0-8c44-f3f929be8e0f") : secret "dns-default-metrics-tls" not found
Jul 10 00:17:33.173162 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:33.173131    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-klmj6"
Jul 10 00:17:48.983150 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:17:48.983108    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:18:00.612027 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:18:00.611970    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs\") pod \"network-metrics-daemon-qzdmp\" (UID: \"1535944a-df71-496e-955d-7d67d827bec0\") " pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:18:00.612457 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:18:00.612128    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 10 00:18:00.612457 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:18:00.612196    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs podName:1535944a-df71-496e-955d-7d67d827bec0 nodeName:}" failed. No retries permitted until 2026-07-10 00:20:02.612179714 +0000 UTC m=+252.314964540 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs") pod "network-metrics-daemon-qzdmp" (UID: "1535944a-df71-496e-955d-7d67d827bec0") : secret "metrics-daemon-secret" not found
Jul 10 00:18:30.139144 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:18:30.139085    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-j8x4j" podUID="6e7a4774-431a-40c0-8c44-f3f929be8e0f"
Jul 10 00:18:30.382835 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:18:30.382803    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:18:30.840381 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:18:30.840343    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-qzdmp" podUID="1535944a-df71-496e-955d-7d67d827bec0"
Jul 10 00:18:35.043659 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:18:35.043608    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:18:35.044054 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:18:35.043762    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 00:18:35.044054 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:18:35.043824    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls podName:6e7a4774-431a-40c0-8c44-f3f929be8e0f nodeName:}" failed. No retries permitted until 2026-07-10 00:20:37.043806448 +0000 UTC m=+286.746591290 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls") pod "dns-default-j8x4j" (UID: "6e7a4774-431a-40c0-8c44-f3f929be8e0f") : secret "dns-default-metrics-tls" not found
Jul 10 00:18:44.830819 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:18:44.830729    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:19:08.039997 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:08.039961    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:24.510444 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:24.510410    2559 generic.go:356] "Generic (PLEG): container finished" podID="88345d5f-b0a0-4fb8-b294-6e47d47424b9" containerID="091edc4b4792e79840f71a5e92437ca2f70765cde679494dbdb18249111601ae" exitCode=255
Jul 10 00:19:24.510816 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:24.510466    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc" event={"ID":"88345d5f-b0a0-4fb8-b294-6e47d47424b9","Type":"ContainerDied","Data":"091edc4b4792e79840f71a5e92437ca2f70765cde679494dbdb18249111601ae"}
Jul 10 00:19:24.515807 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:24.515787    2559 scope.go:117] "RemoveContainer" containerID="091edc4b4792e79840f71a5e92437ca2f70765cde679494dbdb18249111601ae"
Jul 10 00:19:25.514862 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:25.514830    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bfcd4d56-k7bzc" event={"ID":"88345d5f-b0a0-4fb8-b294-6e47d47424b9","Type":"ContainerStarted","Data":"7b97f2b94d5ff8712fdc9d9015630e4120304a41ffbdb04dc3cf66716b06edf3"}
Jul 10 00:19:32.560747 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.560711    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-dnvt2"]
Jul 10 00:19:32.563567 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.563551    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.565233 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.565211    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b8fc76db4-hfjmw"]
Jul 10 00:19:32.566786 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.566763    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 10 00:19:32.566892 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.566791    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 10 00:19:32.566958 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.566946    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 10 00:19:32.567064 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.567046    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-qg8hq\""
Jul 10 00:19:32.567150 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.567135    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 10 00:19:32.567837 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.567823    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.570026 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.570008    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 10 00:19:32.570137 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.570049    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 10 00:19:32.570562 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.570531    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s69wz\""
Jul 10 00:19:32.572337 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.571951    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 10 00:19:32.579767 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.579738    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-dnvt2"]
Jul 10 00:19:32.580178 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.580158    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 10 00:19:32.580258 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.580164    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 10 00:19:32.583338 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.583303    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b8fc76db4-hfjmw"]
Jul 10 00:19:32.618985 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.618963    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db79ea28-a194-4f03-ab3c-a6faee9a011d-tmp\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.619099 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.619008    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ls9n\" (UniqueName: \"kubernetes.io/projected/db79ea28-a194-4f03-ab3c-a6faee9a011d-kube-api-access-9ls9n\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.619099 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.619030    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4703c32b-78f0-4cbc-9461-aa323d3fcf67-image-registry-private-configuration\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.619099 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.619077    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4703c32b-78f0-4cbc-9461-aa323d3fcf67-trusted-ca\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.619266 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.619113    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-bound-sa-token\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.619266 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.619147    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4703c32b-78f0-4cbc-9461-aa323d3fcf67-ca-trust-extracted\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.619266 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.619189    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/db79ea28-a194-4f03-ab3c-a6faee9a011d-snapshots\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.619266 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.619219    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db79ea28-a194-4f03-ab3c-a6faee9a011d-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.619266 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.619247    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2c9v\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-kube-api-access-q2c9v\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.619516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.619284    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-certificates\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.619516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.619308    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4703c32b-78f0-4cbc-9461-aa323d3fcf67-installation-pull-secrets\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.619516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.619364    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.619516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.619390    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db79ea28-a194-4f03-ab3c-a6faee9a011d-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.619516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.619417    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db79ea28-a194-4f03-ab3c-a6faee9a011d-serving-cert\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.656744 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.656721    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82"]
Jul 10 00:19:32.659621 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.659602    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82"
Jul 10 00:19:32.660249 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.660232    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-flrgf"]
Jul 10 00:19:32.662151 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.662130    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 10 00:19:32.662434 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.662415    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 10 00:19:32.662528 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.662441    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 10 00:19:32.662735 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.662723    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-b2bcr\""
Jul 10 00:19:32.662914 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.662899    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 10 00:19:32.663013 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.662998    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"]
Jul 10 00:19:32.663174 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.663100    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-flrgf"
Jul 10 00:19:32.665487 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.665470    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8v7tk\""
Jul 10 00:19:32.665579 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.665553    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 10 00:19:32.665663 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.665645    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 10 00:19:32.665726 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.665686    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w"]
Jul 10 00:19:32.665822 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.665807    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:19:32.668370 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.668355    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w"
Jul 10 00:19:32.668906 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.668891    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82"]
Jul 10 00:19:32.669569 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.669508    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 10 00:19:32.669569 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.669537    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 10 00:19:32.669569 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.669557    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-56fp7\""
Jul 10 00:19:32.669866 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.669691    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 10 00:19:32.669866 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.669799    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 10 00:19:32.670495 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.670479    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 10 00:19:32.670677 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.670648    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 10 00:19:32.670802 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.670785    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 10 00:19:32.670949 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.670934    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-mf6tb\""
Jul 10 00:19:32.674595 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.674495    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-flrgf"]
Jul 10 00:19:32.674815 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.674794    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w"]
Jul 10 00:19:32.685517 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.685497    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"]
Jul 10 00:19:32.719995 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.719966    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbtzq\" (UniqueName: \"kubernetes.io/projected/3480f56e-aaf4-4756-be9d-bd4216f83859-kube-api-access-tbtzq\") pod \"cluster-samples-operator-65db858894-sqm9w\" (UID: \"3480f56e-aaf4-4756-be9d-bd4216f83859\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w"
Jul 10 00:19:32.720095 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720004    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-bound-sa-token\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.720095 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720035    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/db79ea28-a194-4f03-ab3c-a6faee9a011d-snapshots\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.720095 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720077    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4703c32b-78f0-4cbc-9461-aa323d3fcf67-ca-trust-extracted\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.720255 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720101    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-sqm9w\" (UID: \"3480f56e-aaf4-4756-be9d-bd4216f83859\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w"
Jul 10 00:19:32.720255 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720154    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-blndf\" (UID: \"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:19:32.720255 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720181    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcvm2\" (UniqueName: \"kubernetes.io/projected/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-kube-api-access-pcvm2\") pod \"cluster-monitoring-operator-5f7f494dd4-blndf\" (UID: \"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:19:32.720255 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720212    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-certificates\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.720516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720282    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.720516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720355    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db79ea28-a194-4f03-ab3c-a6faee9a011d-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.720516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720396    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db79ea28-a194-4f03-ab3c-a6faee9a011d-serving-cert\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.720516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720430    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0eef5919-029b-4087-93e6-c44b8fdce6fa-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-tsm82\" (UID: \"0eef5919-029b-4087-93e6-c44b8fdce6fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82"
Jul 10 00:19:32.720516 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:32.720449    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 00:19:32.720516 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:32.720466    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8fc76db4-hfjmw: secret "image-registry-tls" not found
Jul 10 00:19:32.720516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720463    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4703c32b-78f0-4cbc-9461-aa323d3fcf67-ca-trust-extracted\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.720850 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:32.720537    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls podName:4703c32b-78f0-4cbc-9461-aa323d3fcf67 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:33.2205222 +0000 UTC m=+222.923307039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls") pod "image-registry-b8fc76db4-hfjmw" (UID: "4703c32b-78f0-4cbc-9461-aa323d3fcf67") : secret "image-registry-tls" not found
Jul 10 00:19:32.720850 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720463    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ls9n\" (UniqueName: \"kubernetes.io/projected/db79ea28-a194-4f03-ab3c-a6faee9a011d-kube-api-access-9ls9n\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.720850 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720612    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/db79ea28-a194-4f03-ab3c-a6faee9a011d-snapshots\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.720850 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720674    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db79ea28-a194-4f03-ab3c-a6faee9a011d-tmp\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.720850 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720695    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4703c32b-78f0-4cbc-9461-aa323d3fcf67-trusted-ca\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.720850 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720724    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0eef5919-029b-4087-93e6-c44b8fdce6fa-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-tsm82\" (UID: \"0eef5919-029b-4087-93e6-c44b8fdce6fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82"
Jul 10 00:19:32.720850 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720775    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4703c32b-78f0-4cbc-9461-aa323d3fcf67-image-registry-private-configuration\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.720850 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720835    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db79ea28-a194-4f03-ab3c-a6faee9a011d-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.721252 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720870    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbwdv\" (UniqueName: \"kubernetes.io/projected/e9477f6c-0ae3-4303-8207-30e23db3c4a3-kube-api-access-fbwdv\") pod \"volume-data-source-validator-78495597f6-flrgf\" (UID: \"e9477f6c-0ae3-4303-8207-30e23db3c4a3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-flrgf"
Jul 10 00:19:32.721252 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720896    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-blndf\" (UID: \"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:19:32.721252 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720924    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2c9v\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-kube-api-access-q2c9v\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.721252 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720948    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db79ea28-a194-4f03-ab3c-a6faee9a011d-tmp\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.721252 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.720951    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4703c32b-78f0-4cbc-9461-aa323d3fcf67-installation-pull-secrets\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.721252 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.721008    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l4v9\" (UniqueName: \"kubernetes.io/projected/0eef5919-029b-4087-93e6-c44b8fdce6fa-kube-api-access-6l4v9\") pod \"kube-storage-version-migrator-operator-799cb7f958-tsm82\" (UID: \"0eef5919-029b-4087-93e6-c44b8fdce6fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82"
Jul 10 00:19:32.721536 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.721270    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-certificates\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.721536 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.721271    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db79ea28-a194-4f03-ab3c-a6faee9a011d-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.721610 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.721597    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db79ea28-a194-4f03-ab3c-a6faee9a011d-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.721709 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.721676    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4703c32b-78f0-4cbc-9461-aa323d3fcf67-trusted-ca\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.722820 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.722802    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db79ea28-a194-4f03-ab3c-a6faee9a011d-serving-cert\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.723089 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.723070    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4703c32b-78f0-4cbc-9461-aa323d3fcf67-image-registry-private-configuration\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.723199 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.723182    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4703c32b-78f0-4cbc-9461-aa323d3fcf67-installation-pull-secrets\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.731215 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.731193    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-bound-sa-token\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.732128 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.732101    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ls9n\" (UniqueName: \"kubernetes.io/projected/db79ea28-a194-4f03-ab3c-a6faee9a011d-kube-api-access-9ls9n\") pod \"insights-operator-68d9dd94cf-dnvt2\" (UID: \"db79ea28-a194-4f03-ab3c-a6faee9a011d\") " pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.732417 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.732401    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2c9v\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-kube-api-access-q2c9v\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:32.758559 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.758534    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-8j7st"]
Jul 10 00:19:32.762606 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.762591    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st"
Jul 10 00:19:32.765372 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.765285    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 10 00:19:32.765516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.765453    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 10 00:19:32.765720 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.765696    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-smbmv\""
Jul 10 00:19:32.778505 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.778481    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-8j7st"]
Jul 10 00:19:32.821560 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.821487    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0eef5919-029b-4087-93e6-c44b8fdce6fa-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-tsm82\" (UID: \"0eef5919-029b-4087-93e6-c44b8fdce6fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82"
Jul 10 00:19:32.821560 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.821526    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/44e54584-138b-4286-a17e-3e16fabe6861-nginx-conf\") pod \"networking-console-plugin-6cf856758c-8j7st\" (UID: \"44e54584-138b-4286-a17e-3e16fabe6861\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st"
Jul 10 00:19:32.821560 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.821552    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8j7st\" (UID: \"44e54584-138b-4286-a17e-3e16fabe6861\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st"
Jul 10 00:19:32.821750 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.821574    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0eef5919-029b-4087-93e6-c44b8fdce6fa-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-tsm82\" (UID: \"0eef5919-029b-4087-93e6-c44b8fdce6fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82"
Jul 10 00:19:32.821750 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.821651    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwdv\" (UniqueName: \"kubernetes.io/projected/e9477f6c-0ae3-4303-8207-30e23db3c4a3-kube-api-access-fbwdv\") pod \"volume-data-source-validator-78495597f6-flrgf\" (UID: \"e9477f6c-0ae3-4303-8207-30e23db3c4a3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-flrgf"
Jul 10 00:19:32.821750 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.821687    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-blndf\" (UID: \"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:19:32.821750 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.821717    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l4v9\" (UniqueName: \"kubernetes.io/projected/0eef5919-029b-4087-93e6-c44b8fdce6fa-kube-api-access-6l4v9\") pod \"kube-storage-version-migrator-operator-799cb7f958-tsm82\" (UID: \"0eef5919-029b-4087-93e6-c44b8fdce6fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82"
Jul 10 00:19:32.821750 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.821742    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbtzq\" (UniqueName: \"kubernetes.io/projected/3480f56e-aaf4-4756-be9d-bd4216f83859-kube-api-access-tbtzq\") pod \"cluster-samples-operator-65db858894-sqm9w\" (UID: \"3480f56e-aaf4-4756-be9d-bd4216f83859\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w"
Jul 10 00:19:32.821976 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.821784    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-sqm9w\" (UID: \"3480f56e-aaf4-4756-be9d-bd4216f83859\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w"
Jul 10 00:19:32.821976 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:32.821893    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 10 00:19:32.821976 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:32.821959    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls podName:3480f56e-aaf4-4756-be9d-bd4216f83859 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:33.321940058 +0000 UTC m=+223.024724889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls") pod "cluster-samples-operator-65db858894-sqm9w" (UID: "3480f56e-aaf4-4756-be9d-bd4216f83859") : secret "samples-operator-tls" not found
Jul 10 00:19:32.822138 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.821976    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-blndf\" (UID: \"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:19:32.822138 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.822004    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcvm2\" (UniqueName: \"kubernetes.io/projected/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-kube-api-access-pcvm2\") pod \"cluster-monitoring-operator-5f7f494dd4-blndf\" (UID: \"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:19:32.822138 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:32.822058    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 00:19:32.822138 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:32.822114    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls podName:a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:33.322098438 +0000 UTC m=+223.024883279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-blndf" (UID: "a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7") : secret "cluster-monitoring-operator-tls" not found
Jul 10 00:19:32.822138 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.822116    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0eef5919-029b-4087-93e6-c44b8fdce6fa-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-tsm82\" (UID: \"0eef5919-029b-4087-93e6-c44b8fdce6fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82"
Jul 10 00:19:32.822577 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.822553    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-blndf\" (UID: \"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:19:32.823853 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.823834    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0eef5919-029b-4087-93e6-c44b8fdce6fa-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-tsm82\" (UID: \"0eef5919-029b-4087-93e6-c44b8fdce6fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82"
Jul 10 00:19:32.832423 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.832399    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcvm2\" (UniqueName: \"kubernetes.io/projected/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-kube-api-access-pcvm2\") pod \"cluster-monitoring-operator-5f7f494dd4-blndf\" (UID: \"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:19:32.832536 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.832477    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbtzq\" (UniqueName: \"kubernetes.io/projected/3480f56e-aaf4-4756-be9d-bd4216f83859-kube-api-access-tbtzq\") pod \"cluster-samples-operator-65db858894-sqm9w\" (UID: \"3480f56e-aaf4-4756-be9d-bd4216f83859\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w"
Jul 10 00:19:32.833514 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.833488    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l4v9\" (UniqueName: \"kubernetes.io/projected/0eef5919-029b-4087-93e6-c44b8fdce6fa-kube-api-access-6l4v9\") pod \"kube-storage-version-migrator-operator-799cb7f958-tsm82\" (UID: \"0eef5919-029b-4087-93e6-c44b8fdce6fa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82"
Jul 10 00:19:32.833585 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.833541    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbwdv\" (UniqueName: \"kubernetes.io/projected/e9477f6c-0ae3-4303-8207-30e23db3c4a3-kube-api-access-fbwdv\") pod \"volume-data-source-validator-78495597f6-flrgf\" (UID: \"e9477f6c-0ae3-4303-8207-30e23db3c4a3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-flrgf"
Jul 10 00:19:32.876203 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.876182    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2"
Jul 10 00:19:32.889582 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.889562    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:32.922764 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.922732    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/44e54584-138b-4286-a17e-3e16fabe6861-nginx-conf\") pod \"networking-console-plugin-6cf856758c-8j7st\" (UID: \"44e54584-138b-4286-a17e-3e16fabe6861\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st"
Jul 10 00:19:32.922911 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.922783    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8j7st\" (UID: \"44e54584-138b-4286-a17e-3e16fabe6861\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st"
Jul 10 00:19:32.923042 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:32.923007    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 00:19:32.923196 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:32.923121    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert podName:44e54584-138b-4286-a17e-3e16fabe6861 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:33.423098375 +0000 UTC m=+223.125883215 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8j7st" (UID: "44e54584-138b-4286-a17e-3e16fabe6861") : secret "networking-console-plugin-cert" not found
Jul 10 00:19:32.923719 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.923679    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/44e54584-138b-4286-a17e-3e16fabe6861-nginx-conf\") pod \"networking-console-plugin-6cf856758c-8j7st\" (UID: \"44e54584-138b-4286-a17e-3e16fabe6861\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st"
Jul 10 00:19:32.970588 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.970561    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82"
Jul 10 00:19:32.977562 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.977539    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-flrgf"
Jul 10 00:19:32.989820 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.989794    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:32.995352 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:32.995330    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-dnvt2"]
Jul 10 00:19:32.999101 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:19:32.999077    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb79ea28_a194_4f03_ab3c_a6faee9a011d.slice/crio-e3414c333271a039fd843febd82e3cc448697fab90ac4fa2ef380b8a6fe9b542 WatchSource:0}: Error finding container e3414c333271a039fd843febd82e3cc448697fab90ac4fa2ef380b8a6fe9b542: Status 404 returned error can't find the container with id e3414c333271a039fd843febd82e3cc448697fab90ac4fa2ef380b8a6fe9b542
Jul 10 00:19:33.106146 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:33.106110    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82"]
Jul 10 00:19:33.110627 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:19:33.110590    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eef5919_029b_4087_93e6_c44b8fdce6fa.slice/crio-b2fbae4bf11f6bf59b2e18cf989dd0a953c70d8e01e449f24edf2969aef0c66e WatchSource:0}: Error finding container b2fbae4bf11f6bf59b2e18cf989dd0a953c70d8e01e449f24edf2969aef0c66e: Status 404 returned error can't find the container with id b2fbae4bf11f6bf59b2e18cf989dd0a953c70d8e01e449f24edf2969aef0c66e
Jul 10 00:19:33.117958 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:33.117934    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-flrgf"]
Jul 10 00:19:33.120677 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:19:33.120656    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9477f6c_0ae3_4303_8207_30e23db3c4a3.slice/crio-eee6571b20fb3b2cd25cdece7938d9ea52e59b1dca8974bcbd25300e79b332e9 WatchSource:0}: Error finding container eee6571b20fb3b2cd25cdece7938d9ea52e59b1dca8974bcbd25300e79b332e9: Status 404 returned error can't find the container with id eee6571b20fb3b2cd25cdece7938d9ea52e59b1dca8974bcbd25300e79b332e9
Jul 10 00:19:33.224786 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:33.224754    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:33.224954 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:33.224890    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 00:19:33.224954 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:33.224904    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8fc76db4-hfjmw: secret "image-registry-tls" not found
Jul 10 00:19:33.225035 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:33.224957    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls podName:4703c32b-78f0-4cbc-9461-aa323d3fcf67 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:34.224942366 +0000 UTC m=+223.927727205 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls") pod "image-registry-b8fc76db4-hfjmw" (UID: "4703c32b-78f0-4cbc-9461-aa323d3fcf67") : secret "image-registry-tls" not found
Jul 10 00:19:33.326034 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:33.325924    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-sqm9w\" (UID: \"3480f56e-aaf4-4756-be9d-bd4216f83859\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w"
Jul 10 00:19:33.326034 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:33.325991    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-blndf\" (UID: \"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:19:33.326234 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:33.326070    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 10 00:19:33.326234 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:33.326114    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 00:19:33.326234 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:33.326138    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls podName:3480f56e-aaf4-4756-be9d-bd4216f83859 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:34.326121294 +0000 UTC m=+224.028906121 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls") pod "cluster-samples-operator-65db858894-sqm9w" (UID: "3480f56e-aaf4-4756-be9d-bd4216f83859") : secret "samples-operator-tls" not found
Jul 10 00:19:33.326234 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:33.326156    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls podName:a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:34.326143406 +0000 UTC m=+224.028928231 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-blndf" (UID: "a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7") : secret "cluster-monitoring-operator-tls" not found
Jul 10 00:19:33.427005 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:33.426969    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8j7st\" (UID: \"44e54584-138b-4286-a17e-3e16fabe6861\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st"
Jul 10 00:19:33.427170 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:33.427107    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 00:19:33.427209 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:33.427170    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert podName:44e54584-138b-4286-a17e-3e16fabe6861 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:34.427153608 +0000 UTC m=+224.129938439 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8j7st" (UID: "44e54584-138b-4286-a17e-3e16fabe6861") : secret "networking-console-plugin-cert" not found
Jul 10 00:19:33.538429 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:33.538394    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82" event={"ID":"0eef5919-029b-4087-93e6-c44b8fdce6fa","Type":"ContainerStarted","Data":"b2fbae4bf11f6bf59b2e18cf989dd0a953c70d8e01e449f24edf2969aef0c66e"}
Jul 10 00:19:33.539365 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:33.539344    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2" event={"ID":"db79ea28-a194-4f03-ab3c-a6faee9a011d","Type":"ContainerStarted","Data":"e3414c333271a039fd843febd82e3cc448697fab90ac4fa2ef380b8a6fe9b542"}
Jul 10 00:19:33.540237 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:33.540218    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-flrgf" event={"ID":"e9477f6c-0ae3-4303-8207-30e23db3c4a3","Type":"ContainerStarted","Data":"eee6571b20fb3b2cd25cdece7938d9ea52e59b1dca8974bcbd25300e79b332e9"}
Jul 10 00:19:34.234666 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:34.234624    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:34.235136 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:34.234780    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 00:19:34.235136 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:34.234808    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8fc76db4-hfjmw: secret "image-registry-tls" not found
Jul 10 00:19:34.235136 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:34.234882    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls podName:4703c32b-78f0-4cbc-9461-aa323d3fcf67 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:36.234861305 +0000 UTC m=+225.937646152 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls") pod "image-registry-b8fc76db4-hfjmw" (UID: "4703c32b-78f0-4cbc-9461-aa323d3fcf67") : secret "image-registry-tls" not found
Jul 10 00:19:34.335525 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:34.335484    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-sqm9w\" (UID: \"3480f56e-aaf4-4756-be9d-bd4216f83859\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w"
Jul 10 00:19:34.335692 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:34.335555    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-blndf\" (UID: \"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:19:34.335797 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:34.335754    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 10 00:19:34.335851 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:34.335792    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 00:19:34.335851 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:34.335840    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls podName:3480f56e-aaf4-4756-be9d-bd4216f83859 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:36.335814749 +0000 UTC m=+226.038599598 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls") pod "cluster-samples-operator-65db858894-sqm9w" (UID: "3480f56e-aaf4-4756-be9d-bd4216f83859") : secret "samples-operator-tls" not found
Jul 10 00:19:34.335959 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:34.335860    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls podName:a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:36.33585109 +0000 UTC m=+226.038635923 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-blndf" (UID: "a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7") : secret "cluster-monitoring-operator-tls" not found
Jul 10 00:19:34.436221 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:34.436181    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8j7st\" (UID: \"44e54584-138b-4286-a17e-3e16fabe6861\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st"
Jul 10 00:19:34.436441 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:34.436395    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 00:19:34.436512 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:34.436482    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert podName:44e54584-138b-4286-a17e-3e16fabe6861 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:36.436460666 +0000 UTC m=+226.139245511 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8j7st" (UID: "44e54584-138b-4286-a17e-3e16fabe6861") : secret "networking-console-plugin-cert" not found
Jul 10 00:19:35.548081 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:35.548042    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-flrgf" event={"ID":"e9477f6c-0ae3-4303-8207-30e23db3c4a3","Type":"ContainerStarted","Data":"b45650b78b15943a8870c8a6bfcdf5969c1f96535bc03f10e7e9a9690756be22"}
Jul 10 00:19:35.562922 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:35.562867    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-flrgf" podStartSLOduration=2.110147618 podStartE2EDuration="3.56284773s" totalImagesPullingTime="1.452700112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:19:35.561827017 +0000 UTC m=+225.264611882" watchObservedRunningTime="2026-07-10 00:19:35.56284773 +0000 UTC m=+225.265632558"
Jul 10 00:19:35.639926 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:35.639873    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:36.250730 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.250685    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:36.250929 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:36.250864    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 00:19:36.250929 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:36.250889    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8fc76db4-hfjmw: secret "image-registry-tls" not found
Jul 10 00:19:36.251029 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:36.250971    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls podName:4703c32b-78f0-4cbc-9461-aa323d3fcf67 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:40.250950063 +0000 UTC m=+229.953734911 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls") pod "image-registry-b8fc76db4-hfjmw" (UID: "4703c32b-78f0-4cbc-9461-aa323d3fcf67") : secret "image-registry-tls" not found
Jul 10 00:19:36.352095 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.352051    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-sqm9w\" (UID: \"3480f56e-aaf4-4756-be9d-bd4216f83859\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w"
Jul 10 00:19:36.352268 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.352108    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-blndf\" (UID: \"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:19:36.352268 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:36.352230    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 10 00:19:36.352378 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:36.352233    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 00:19:36.352378 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:36.352328    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls podName:3480f56e-aaf4-4756-be9d-bd4216f83859 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:40.352287888 +0000 UTC m=+230.055072716 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls") pod "cluster-samples-operator-65db858894-sqm9w" (UID: "3480f56e-aaf4-4756-be9d-bd4216f83859") : secret "samples-operator-tls" not found
Jul 10 00:19:36.352469 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:36.352389    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls podName:a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:40.352371164 +0000 UTC m=+230.055155994 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-blndf" (UID: "a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7") : secret "cluster-monitoring-operator-tls" not found
Jul 10 00:19:36.453243 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.453200    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8j7st\" (UID: \"44e54584-138b-4286-a17e-3e16fabe6861\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st"
Jul 10 00:19:36.453400 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:36.453370    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 00:19:36.453446 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:36.453435    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert podName:44e54584-138b-4286-a17e-3e16fabe6861 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:40.453420629 +0000 UTC m=+230.156205455 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8j7st" (UID: "44e54584-138b-4286-a17e-3e16fabe6861") : secret "networking-console-plugin-cert" not found
Jul 10 00:19:36.552930 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.552834    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82" event={"ID":"0eef5919-029b-4087-93e6-c44b8fdce6fa","Type":"ContainerStarted","Data":"d5b1232befe6f06c84a65953bceda7a9886a4f0152b907c116e599ae19fe3798"}
Jul 10 00:19:36.554466 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.554437    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2" event={"ID":"db79ea28-a194-4f03-ab3c-a6faee9a011d","Type":"ContainerStarted","Data":"f6a5d404fe83ad496864c5e4c08f87ea69196ea3451af594ea2233d8c9e21000"}
Jul 10 00:19:36.569241 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.569194    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82" podStartSLOduration=1.810704185 podStartE2EDuration="4.569178988s" totalImagesPullingTime="2.758474803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:19:36.568555799 +0000 UTC m=+226.271340648" watchObservedRunningTime="2026-07-10 00:19:36.569178988 +0000 UTC m=+226.271963835"
Jul 10 00:19:36.583569 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.583509    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2" podStartSLOduration=1.723476424 podStartE2EDuration="4.58349506s" totalImagesPullingTime="2.860018636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:19:36.583076138 +0000 UTC m=+226.285861006" watchObservedRunningTime="2026-07-10 00:19:36.58349506 +0000 UTC m=+226.286279909"
Jul 10 00:19:36.677489 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.677459    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zdkv9"]
Jul 10 00:19:36.680788 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.680767    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:36.684052 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.684027    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 10 00:19:36.685057 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.685034    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2znzl\""
Jul 10 00:19:36.685165 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.685077    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 10 00:19:36.705061 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.705034    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zdkv9"]
Jul 10 00:19:36.734703 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.734673    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:36.756087 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.756058    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ed80c3b6-d4f4-490e-a89e-874824a1a195-crio-socket\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:36.756296 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.756139    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:36.756296 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.756192    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6xgs\" (UniqueName: \"kubernetes.io/projected/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-api-access-w6xgs\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:36.756296 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.756255    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:36.756450 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.756295    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed80c3b6-d4f4-490e-a89e-874824a1a195-data-volume\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:36.800541 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.800484    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:36.857114 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.857033    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:36.857114 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.857080    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6xgs\" (UniqueName: \"kubernetes.io/projected/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-api-access-w6xgs\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:36.857356 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:36.857164    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 00:19:36.857356 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.857165    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:36.857356 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:36.857238    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 00:19:36.857356 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:36.857241    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm podName:ed80c3b6-d4f4-490e-a89e-874824a1a195 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:37.357217372 +0000 UTC m=+227.060002200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zdkv9" (UID: "ed80c3b6-d4f4-490e-a89e-874824a1a195") : configmap "kube-rbac-proxy" not found
Jul 10 00:19:36.857356 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:36.857346    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls podName:ed80c3b6-d4f4-490e-a89e-874824a1a195 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:37.357299417 +0000 UTC m=+227.060084245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zdkv9" (UID: "ed80c3b6-d4f4-490e-a89e-874824a1a195") : secret "insights-runtime-extractor-tls" not found
Jul 10 00:19:36.857543 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.857417    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed80c3b6-d4f4-490e-a89e-874824a1a195-data-volume\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:36.857543 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.857499    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ed80c3b6-d4f4-490e-a89e-874824a1a195-crio-socket\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:36.857628 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.857596    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ed80c3b6-d4f4-490e-a89e-874824a1a195-crio-socket\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:36.857834 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.857814    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed80c3b6-d4f4-490e-a89e-874824a1a195-data-volume\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:36.871185 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:36.871154    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6xgs\" (UniqueName: \"kubernetes.io/projected/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-api-access-w6xgs\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:37.362472 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:37.362434    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:37.362664 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:37.362501    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:37.362664 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:37.362559    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 00:19:37.362664 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:37.362622    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 00:19:37.362664 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:37.362625    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm podName:ed80c3b6-d4f4-490e-a89e-874824a1a195 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:38.362610128 +0000 UTC m=+228.065394954 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zdkv9" (UID: "ed80c3b6-d4f4-490e-a89e-874824a1a195") : configmap "kube-rbac-proxy" not found
Jul 10 00:19:37.362793 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:37.362674    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls podName:ed80c3b6-d4f4-490e-a89e-874824a1a195 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:38.36266003 +0000 UTC m=+228.065444857 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zdkv9" (UID: "ed80c3b6-d4f4-490e-a89e-874824a1a195") : secret "insights-runtime-extractor-tls" not found
Jul 10 00:19:37.466712 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:37.466675    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:37.533443 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:37.533406    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:37.592899 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:37.592854    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:37.769249 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:37.769216    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:38.248376 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:38.248342    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:38.300378 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:38.300346    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:38.371151 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:38.371116    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:38.371339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:38.371174    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:38.371339 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:38.371249    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 00:19:38.371339 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:38.371267    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 00:19:38.371339 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:38.371337    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls podName:ed80c3b6-d4f4-490e-a89e-874824a1a195 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:40.371303051 +0000 UTC m=+230.074087893 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zdkv9" (UID: "ed80c3b6-d4f4-490e-a89e-874824a1a195") : secret "insights-runtime-extractor-tls" not found
Jul 10 00:19:38.371473 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:38.371356    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm podName:ed80c3b6-d4f4-490e-a89e-874824a1a195 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:40.37134798 +0000 UTC m=+230.074132805 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zdkv9" (UID: "ed80c3b6-d4f4-490e-a89e-874824a1a195") : configmap "kube-rbac-proxy" not found
Jul 10 00:19:38.861619 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:38.861588    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wtmzf_bc636ec8-6508-407c-8f57-38cf0bcf86a3/dns-node-resolver/0.log"
Jul 10 00:19:39.582685 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:39.582656    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:39.635224 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:39.635189    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:39.845210 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:39.845135    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-jzm6s"]
Jul 10 00:19:39.848283 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:39.848268    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-jzm6s"
Jul 10 00:19:39.850867 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:39.850839    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 10 00:19:39.852111 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:39.852086    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qt49d\""
Jul 10 00:19:39.852211 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:39.852123    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 10 00:19:39.852211 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:39.852151    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 10 00:19:39.852211 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:39.852152    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 10 00:19:39.856278 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:39.856257    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-jzm6s"]
Jul 10 00:19:39.863753 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:39.863738    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gphtp_7ba31ebb-c51d-4692-80de-d8a7384b7e2c/node-ca/0.log"
Jul 10 00:19:39.878778 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:39.878753    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:39.930301 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:39.930275    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:39.984767 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:39.984736    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/61da2195-27ff-4dcf-b1be-7a8ceef86c72-signing-key\") pod \"service-ca-79f68b9f4f-jzm6s\" (UID: \"61da2195-27ff-4dcf-b1be-7a8ceef86c72\") " pod="openshift-service-ca/service-ca-79f68b9f4f-jzm6s"
Jul 10 00:19:39.984892 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:39.984793    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzqcd\" (UniqueName: \"kubernetes.io/projected/61da2195-27ff-4dcf-b1be-7a8ceef86c72-kube-api-access-hzqcd\") pod \"service-ca-79f68b9f4f-jzm6s\" (UID: \"61da2195-27ff-4dcf-b1be-7a8ceef86c72\") " pod="openshift-service-ca/service-ca-79f68b9f4f-jzm6s"
Jul 10 00:19:39.984892 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:39.984829    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/61da2195-27ff-4dcf-b1be-7a8ceef86c72-signing-cabundle\") pod \"service-ca-79f68b9f4f-jzm6s\" (UID: \"61da2195-27ff-4dcf-b1be-7a8ceef86c72\") " pod="openshift-service-ca/service-ca-79f68b9f4f-jzm6s"
Jul 10 00:19:40.085415 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.085372    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/61da2195-27ff-4dcf-b1be-7a8ceef86c72-signing-key\") pod \"service-ca-79f68b9f4f-jzm6s\" (UID: \"61da2195-27ff-4dcf-b1be-7a8ceef86c72\") " pod="openshift-service-ca/service-ca-79f68b9f4f-jzm6s"
Jul 10 00:19:40.085553 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.085444    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzqcd\" (UniqueName: \"kubernetes.io/projected/61da2195-27ff-4dcf-b1be-7a8ceef86c72-kube-api-access-hzqcd\") pod \"service-ca-79f68b9f4f-jzm6s\" (UID: \"61da2195-27ff-4dcf-b1be-7a8ceef86c72\") " pod="openshift-service-ca/service-ca-79f68b9f4f-jzm6s"
Jul 10 00:19:40.085553 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.085471    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/61da2195-27ff-4dcf-b1be-7a8ceef86c72-signing-cabundle\") pod \"service-ca-79f68b9f4f-jzm6s\" (UID: \"61da2195-27ff-4dcf-b1be-7a8ceef86c72\") " pod="openshift-service-ca/service-ca-79f68b9f4f-jzm6s"
Jul 10 00:19:40.086095 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.086077    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/61da2195-27ff-4dcf-b1be-7a8ceef86c72-signing-cabundle\") pod \"service-ca-79f68b9f4f-jzm6s\" (UID: \"61da2195-27ff-4dcf-b1be-7a8ceef86c72\") " pod="openshift-service-ca/service-ca-79f68b9f4f-jzm6s"
Jul 10 00:19:40.087704 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.087682    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/61da2195-27ff-4dcf-b1be-7a8ceef86c72-signing-key\") pod \"service-ca-79f68b9f4f-jzm6s\" (UID: \"61da2195-27ff-4dcf-b1be-7a8ceef86c72\") " pod="openshift-service-ca/service-ca-79f68b9f4f-jzm6s"
Jul 10 00:19:40.093537 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.093514    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzqcd\" (UniqueName: \"kubernetes.io/projected/61da2195-27ff-4dcf-b1be-7a8ceef86c72-kube-api-access-hzqcd\") pod \"service-ca-79f68b9f4f-jzm6s\" (UID: \"61da2195-27ff-4dcf-b1be-7a8ceef86c72\") " pod="openshift-service-ca/service-ca-79f68b9f4f-jzm6s"
Jul 10 00:19:40.157867 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.157775    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-jzm6s"
Jul 10 00:19:40.277079 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.277048    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-jzm6s"]
Jul 10 00:19:40.280536 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:19:40.280509    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61da2195_27ff_4dcf_b1be_7a8ceef86c72.slice/crio-f077ef4ae2462c305839e368c79939820a8f722c86eb06757475159045783c3d WatchSource:0}: Error finding container f077ef4ae2462c305839e368c79939820a8f722c86eb06757475159045783c3d: Status 404 returned error can't find the container with id f077ef4ae2462c305839e368c79939820a8f722c86eb06757475159045783c3d
Jul 10 00:19:40.287229 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.287206    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:40.287422 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:40.287403    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 00:19:40.287477 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:40.287422    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8fc76db4-hfjmw: secret "image-registry-tls" not found
Jul 10 00:19:40.287512 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:40.287486    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls podName:4703c32b-78f0-4cbc-9461-aa323d3fcf67 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:48.287470208 +0000 UTC m=+237.990255034 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls") pod "image-registry-b8fc76db4-hfjmw" (UID: "4703c32b-78f0-4cbc-9461-aa323d3fcf67") : secret "image-registry-tls" not found
Jul 10 00:19:40.388581 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.388543    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-blndf\" (UID: \"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:19:40.388759 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.388598    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:40.388759 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.388646    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:40.388759 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.388684    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-sqm9w\" (UID: \"3480f56e-aaf4-4756-be9d-bd4216f83859\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w"
Jul 10 00:19:40.388759 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:40.388703    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 00:19:40.388759 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:40.388725    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 00:19:40.388933 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:40.388770    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 10 00:19:40.388933 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:40.388772    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 00:19:40.388933 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:40.388778    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls podName:a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:48.388758278 +0000 UTC m=+238.091543110 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-blndf" (UID: "a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7") : secret "cluster-monitoring-operator-tls" not found
Jul 10 00:19:40.388933 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:40.388837    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm podName:ed80c3b6-d4f4-490e-a89e-874824a1a195 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:44.388824247 +0000 UTC m=+234.091609073 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zdkv9" (UID: "ed80c3b6-d4f4-490e-a89e-874824a1a195") : configmap "kube-rbac-proxy" not found
Jul 10 00:19:40.388933 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:40.388849    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls podName:3480f56e-aaf4-4756-be9d-bd4216f83859 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:48.388842977 +0000 UTC m=+238.091627803 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls") pod "cluster-samples-operator-65db858894-sqm9w" (UID: "3480f56e-aaf4-4756-be9d-bd4216f83859") : secret "samples-operator-tls" not found
Jul 10 00:19:40.388933 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:40.388859    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls podName:ed80c3b6-d4f4-490e-a89e-874824a1a195 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:44.388853426 +0000 UTC m=+234.091638252 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zdkv9" (UID: "ed80c3b6-d4f4-490e-a89e-874824a1a195") : secret "insights-runtime-extractor-tls" not found
Jul 10 00:19:40.489269 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.489231    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8j7st\" (UID: \"44e54584-138b-4286-a17e-3e16fabe6861\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st"
Jul 10 00:19:40.489442 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:40.489395    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 00:19:40.489484 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:40.489453    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert podName:44e54584-138b-4286-a17e-3e16fabe6861 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:48.489438962 +0000 UTC m=+238.192223788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8j7st" (UID: "44e54584-138b-4286-a17e-3e16fabe6861") : secret "networking-console-plugin-cert" not found
Jul 10 00:19:40.572730 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.572693    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-jzm6s" event={"ID":"61da2195-27ff-4dcf-b1be-7a8ceef86c72","Type":"ContainerStarted","Data":"f077ef4ae2462c305839e368c79939820a8f722c86eb06757475159045783c3d"}
Jul 10 00:19:40.662722 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.662695    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vtm7t_1d2b0c9d-41f9-45be-a60c-52fa7cb19f15/serve-healthcheck-canary/0.log"
Jul 10 00:19:40.712881 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:40.712848    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:43.079007 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:43.078970    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:43.583205 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:43.583162    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-jzm6s" event={"ID":"61da2195-27ff-4dcf-b1be-7a8ceef86c72","Type":"ContainerStarted","Data":"a7c0bf035904f41b00b5bb8ab3d03558cfc68fd1d30d6c1f5b2f292f5b25a77b"}
Jul 10 00:19:43.600948 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:43.600896    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-jzm6s" podStartSLOduration=2.296326766 podStartE2EDuration="4.600875287s" totalImagesPullingTime="2.304548521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:19:43.59981686 +0000 UTC m=+233.302601708" watchObservedRunningTime="2026-07-10 00:19:43.600875287 +0000 UTC m=+233.303660132"
Jul 10 00:19:44.265894 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:44.265854    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:44.421876 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:44.421831    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:44.423884 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:44.423858    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:44.423969 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:44.423923    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:44.424027 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:44.423988    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 00:19:44.424027 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:44.424024    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 00:19:44.424125 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:44.424063    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm podName:ed80c3b6-d4f4-490e-a89e-874824a1a195 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:52.424044384 +0000 UTC m=+242.126829228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zdkv9" (UID: "ed80c3b6-d4f4-490e-a89e-874824a1a195") : configmap "kube-rbac-proxy" not found
Jul 10 00:19:44.424125 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:44.424099    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls podName:ed80c3b6-d4f4-490e-a89e-874824a1a195 nodeName:}" failed. No retries permitted until 2026-07-10 00:19:52.424089049 +0000 UTC m=+242.126873875 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zdkv9" (UID: "ed80c3b6-d4f4-490e-a89e-874824a1a195") : secret "insights-runtime-extractor-tls" not found
Jul 10 00:19:44.532944 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:44.532861    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:44.680866 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:44.680830    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:44.788834 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:44.788739    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:44.839366 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:44.839304    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:44.950392 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:44.950362    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:45.760244 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:45.760210    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:46.346007 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:46.345976    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:46.554734 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:46.554698    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:46.945270 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:46.945227    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:47.149887 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:47.149853    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:47.437101 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:47.437019    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:47.749971 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:47.749938    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:48.143229 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:48.143154    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:48.354829 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:48.354797    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:48.359419 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:48.359396    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:48.361768 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:48.361748    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls\") pod \"image-registry-b8fc76db4-hfjmw\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") " pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:48.455237 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:48.455208    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:48.460439 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:48.460415    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-sqm9w\" (UID: \"3480f56e-aaf4-4756-be9d-bd4216f83859\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w"
Jul 10 00:19:48.460563 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:48.460468    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-blndf\" (UID: \"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:19:48.460622 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:48.460584    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 00:19:48.460675 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:48.460657    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls podName:a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7 nodeName:}" failed. No retries permitted until 2026-07-10 00:20:04.460637544 +0000 UTC m=+254.163422390 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-blndf" (UID: "a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7") : secret "cluster-monitoring-operator-tls" not found
Jul 10 00:19:48.462814 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:48.462793    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3480f56e-aaf4-4756-be9d-bd4216f83859-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-sqm9w\" (UID: \"3480f56e-aaf4-4756-be9d-bd4216f83859\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w"
Jul 10 00:19:48.481073 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:48.481047    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:48.555549 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:48.555524    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:48.562076 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:48.562044    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8j7st\" (UID: \"44e54584-138b-4286-a17e-3e16fabe6861\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st"
Jul 10 00:19:48.562201 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:48.562184    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 00:19:48.562992 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:48.562968    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert podName:44e54584-138b-4286-a17e-3e16fabe6861 nodeName:}" failed. No retries permitted until 2026-07-10 00:20:04.562940925 +0000 UTC m=+254.265725768 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8j7st" (UID: "44e54584-138b-4286-a17e-3e16fabe6861") : secret "networking-console-plugin-cert" not found
Jul 10 00:19:48.590296 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:48.590266    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w"
Jul 10 00:19:48.613467 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:48.613442    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b8fc76db4-hfjmw"]
Jul 10 00:19:48.616842 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:19:48.616812    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4703c32b_78f0_4cbc_9461_aa323d3fcf67.slice/crio-052f50b9c08ad59e292507da4be48d7fde5e55ab9c4a5b8d86215bbcb709e069 WatchSource:0}: Error finding container 052f50b9c08ad59e292507da4be48d7fde5e55ab9c4a5b8d86215bbcb709e069: Status 404 returned error can't find the container with id 052f50b9c08ad59e292507da4be48d7fde5e55ab9c4a5b8d86215bbcb709e069
Jul 10 00:19:48.724380 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:48.724284    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w"]
Jul 10 00:19:48.962210 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:48.962176    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:49.151156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:49.151074    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:49.348080 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:49.348051    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:49.605905 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:49.605863    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w" event={"ID":"3480f56e-aaf4-4756-be9d-bd4216f83859","Type":"ContainerStarted","Data":"2cd431db5d52114ef16704e575ccf58995bdce07b731d7250aa624ba65d9f709"}
Jul 10 00:19:49.607433 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:49.607409    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw" event={"ID":"4703c32b-78f0-4cbc-9461-aa323d3fcf67","Type":"ContainerStarted","Data":"f12243e2597edbbb22bde40a6ca51baed98a98cccbde274a01c3e148d6d85997"}
Jul 10 00:19:49.607552 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:49.607454    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw" event={"ID":"4703c32b-78f0-4cbc-9461-aa323d3fcf67","Type":"ContainerStarted","Data":"052f50b9c08ad59e292507da4be48d7fde5e55ab9c4a5b8d86215bbcb709e069"}
Jul 10 00:19:49.607552 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:49.607526    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:19:49.626810 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:49.626770    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw" podStartSLOduration=17.626755996 podStartE2EDuration="17.626755996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:19:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:19:49.625630109 +0000 UTC m=+239.328414978" watchObservedRunningTime="2026-07-10 00:19:49.626755996 +0000 UTC m=+239.329540844"
Jul 10 00:19:49.894721 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:49.894648    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:50.347910 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:50.347856    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:50.547969 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:50.547933    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:50.773762 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:50.773728    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:50.974950 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:50.974865    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:51.148219 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:51.148182    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:51.615185 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:51.615151    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w" event={"ID":"3480f56e-aaf4-4756-be9d-bd4216f83859","Type":"ContainerStarted","Data":"881cd6219f8734780b84228a0552dc4ae6d8ed03188b00f8b1ec2556b2ea88e2"}
Jul 10 00:19:51.615656 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:51.615215    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w" event={"ID":"3480f56e-aaf4-4756-be9d-bd4216f83859","Type":"ContainerStarted","Data":"34d782589ee3e108cd15d7b665f78874226beb9eedc4c06681dce7708e642faf"}
Jul 10 00:19:51.631204 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:51.631152    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-sqm9w" podStartSLOduration=17.706782051 podStartE2EDuration="19.631137167s" totalImagesPullingTime="1.924355116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:19:51.629776773 +0000 UTC m=+241.332561622" watchObservedRunningTime="2026-07-10 00:19:51.631137167 +0000 UTC m=+241.333922015"
Jul 10 00:19:51.746521 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:51.746486    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:52.145528 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:52.145491    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:52.498382 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:52.498338    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:52.498550 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:52.498424    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:52.498598 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:52.498548    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 00:19:52.498633 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:19:52.498619    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls podName:ed80c3b6-d4f4-490e-a89e-874824a1a195 nodeName:}" failed. No retries permitted until 2026-07-10 00:20:08.498598561 +0000 UTC m=+258.201383387 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zdkv9" (UID: "ed80c3b6-d4f4-490e-a89e-874824a1a195") : secret "insights-runtime-extractor-tls" not found
Jul 10 00:19:52.498977 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:52.498953    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed80c3b6-d4f4-490e-a89e-874824a1a195-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:19:52.545692 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:52.545661    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:52.746819 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:52.746782    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:53.348612 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:53.348572    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:53.950847 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:53.950808    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:54.147679 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:54.147644    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:54.248958 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:54.248873    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:54.350374 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:54.350343    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:54.401700 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:54.401667    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:54.745203 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:54.745171    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:54.949650 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:54.949621    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:57.049870 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:57.049827    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:57.151833 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:57.151796    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:57.340568 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:57.340482    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:57.721150 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:57.721117    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:58.121093 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:58.120997    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:58.222084 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:58.222056    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:58.322837 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:58.322803    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:58.387517 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:58.387444    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:58.739297 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:58.739263    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:58.928971 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:58.928941    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:59.270366 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:59.270329    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:59.549353 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:59.549262    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:19:59.604600 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:19:59.604574    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:00.570503 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:00.570455    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b8fc76db4-hfjmw"]
Jul 10 00:20:00.583158 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:00.583129    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-cgdmm"]
Jul 10 00:20:00.588871 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:00.588850    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-cgdmm"
Jul 10 00:20:00.591672 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:00.591648    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-l79br\""
Jul 10 00:20:00.591787 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:00.591703    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 10 00:20:00.591787 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:00.591704    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 10 00:20:00.599249 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:00.599231    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-cgdmm"]
Jul 10 00:20:00.681059 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:00.681026    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:00.771490 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:00.771455    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klmwh\" (UniqueName: \"kubernetes.io/projected/0f28bafe-bd50-4228-aee1-04e545bbf94a-kube-api-access-klmwh\") pod \"downloads-68c9cbb497-cgdmm\" (UID: \"0f28bafe-bd50-4228-aee1-04e545bbf94a\") " pod="openshift-console/downloads-68c9cbb497-cgdmm"
Jul 10 00:20:00.779769 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:00.779745    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:00.872302 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:00.872215    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klmwh\" (UniqueName: \"kubernetes.io/projected/0f28bafe-bd50-4228-aee1-04e545bbf94a-kube-api-access-klmwh\") pod \"downloads-68c9cbb497-cgdmm\" (UID: \"0f28bafe-bd50-4228-aee1-04e545bbf94a\") " pod="openshift-console/downloads-68c9cbb497-cgdmm"
Jul 10 00:20:00.877932 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:00.877903    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:00.881598 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:00.881580    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klmwh\" (UniqueName: \"kubernetes.io/projected/0f28bafe-bd50-4228-aee1-04e545bbf94a-kube-api-access-klmwh\") pod \"downloads-68c9cbb497-cgdmm\" (UID: \"0f28bafe-bd50-4228-aee1-04e545bbf94a\") " pod="openshift-console/downloads-68c9cbb497-cgdmm"
Jul 10 00:20:00.897634 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:00.897611    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-cgdmm"
Jul 10 00:20:01.043379 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:01.043254    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-cgdmm"]
Jul 10 00:20:01.045983 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:20:01.045958    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f28bafe_bd50_4228_aee1_04e545bbf94a.slice/crio-bfc534a54eba9885a7cae6b22a6e2dfed51ae41affb2dd1c50979bc66e4a2b99 WatchSource:0}: Error finding container bfc534a54eba9885a7cae6b22a6e2dfed51ae41affb2dd1c50979bc66e4a2b99: Status 404 returned error can't find the container with id bfc534a54eba9885a7cae6b22a6e2dfed51ae41affb2dd1c50979bc66e4a2b99
Jul 10 00:20:01.646779 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:01.646743    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-cgdmm" event={"ID":"0f28bafe-bd50-4228-aee1-04e545bbf94a","Type":"ContainerStarted","Data":"bfc534a54eba9885a7cae6b22a6e2dfed51ae41affb2dd1c50979bc66e4a2b99"}
Jul 10 00:20:02.684420 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:02.684384    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs\") pod \"network-metrics-daemon-qzdmp\" (UID: \"1535944a-df71-496e-955d-7d67d827bec0\") " pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:20:02.687057 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:02.687030    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1535944a-df71-496e-955d-7d67d827bec0-metrics-certs\") pod \"network-metrics-daemon-qzdmp\" (UID: \"1535944a-df71-496e-955d-7d67d827bec0\") " pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:20:02.834105 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:02.834078    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qn6qf\""
Jul 10 00:20:02.842176 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:02.842159    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzdmp"
Jul 10 00:20:02.976917 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:02.976877    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qzdmp"]
Jul 10 00:20:02.983038 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:20:02.983012    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1535944a_df71_496e_955d_7d67d827bec0.slice/crio-1c70e4cbe407041f35150ea9ff02ef5c6982aca67470255f8afb4bff497f080c WatchSource:0}: Error finding container 1c70e4cbe407041f35150ea9ff02ef5c6982aca67470255f8afb4bff497f080c: Status 404 returned error can't find the container with id 1c70e4cbe407041f35150ea9ff02ef5c6982aca67470255f8afb4bff497f080c
Jul 10 00:20:03.653796 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:03.653759    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qzdmp" event={"ID":"1535944a-df71-496e-955d-7d67d827bec0","Type":"ContainerStarted","Data":"1c70e4cbe407041f35150ea9ff02ef5c6982aca67470255f8afb4bff497f080c"}
Jul 10 00:20:04.500636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:04.500596    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-blndf\" (UID: \"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:20:04.503366 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:04.503346    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-blndf\" (UID: \"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:20:04.601791 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:04.601752    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8j7st\" (UID: \"44e54584-138b-4286-a17e-3e16fabe6861\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st"
Jul 10 00:20:04.604436 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:04.604406    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/44e54584-138b-4286-a17e-3e16fabe6861-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8j7st\" (UID: \"44e54584-138b-4286-a17e-3e16fabe6861\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st"
Jul 10 00:20:04.659244 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:04.659209    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qzdmp" event={"ID":"1535944a-df71-496e-955d-7d67d827bec0","Type":"ContainerStarted","Data":"f5153aa667a30d566ee679521a2d90694c6760513789a3e8d7b94726749f6369"}
Jul 10 00:20:04.659404 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:04.659290    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qzdmp" event={"ID":"1535944a-df71-496e-955d-7d67d827bec0","Type":"ContainerStarted","Data":"a81b655bcfaeb48142d29b27f691aeac279f7ca4149e75266926e7b009c2c8fa"}
Jul 10 00:20:04.675659 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:04.675614    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qzdmp" podStartSLOduration=252.597826766 podStartE2EDuration="4m13.675598019s" totalImagesPullingTime="1.077771253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:20:04.674644579 +0000 UTC m=+254.377429460" watchObservedRunningTime="2026-07-10 00:20:04.675598019 +0000 UTC m=+254.378382866"
Jul 10 00:20:04.752049 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:04.751950    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:04.787635 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:04.787612    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-56fp7\""
Jul 10 00:20:04.795721 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:04.795704    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"
Jul 10 00:20:04.804648 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:04.804601    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:04.877889 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:04.877631    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-smbmv\""
Jul 10 00:20:04.884583 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:04.884271    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st"
Jul 10 00:20:04.959706 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:04.959423    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf"]
Jul 10 00:20:04.962736 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:20:04.962700    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2e6f08b_f22e_4ee7_b25d_d84eb08fb4f7.slice/crio-bb6c0c8faa7a1b73b3e23e99878183090231cc8939765d8ed879d038eb6a106a WatchSource:0}: Error finding container bb6c0c8faa7a1b73b3e23e99878183090231cc8939765d8ed879d038eb6a106a: Status 404 returned error can't find the container with id bb6c0c8faa7a1b73b3e23e99878183090231cc8939765d8ed879d038eb6a106a
Jul 10 00:20:05.044019 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:05.043989    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-8j7st"]
Jul 10 00:20:05.046144 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:20:05.046120    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44e54584_138b_4286_a17e_3e16fabe6861.slice/crio-54b7a88a7b657f599c25d9e12afb2ec1290383c6c993f191d7218dd74ee57ee3 WatchSource:0}: Error finding container 54b7a88a7b657f599c25d9e12afb2ec1290383c6c993f191d7218dd74ee57ee3: Status 404 returned error can't find the container with id 54b7a88a7b657f599c25d9e12afb2ec1290383c6c993f191d7218dd74ee57ee3
Jul 10 00:20:05.664913 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:05.664875    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st" event={"ID":"44e54584-138b-4286-a17e-3e16fabe6861","Type":"ContainerStarted","Data":"54b7a88a7b657f599c25d9e12afb2ec1290383c6c993f191d7218dd74ee57ee3"}
Jul 10 00:20:05.666263 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:05.666204    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf" event={"ID":"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7","Type":"ContainerStarted","Data":"bb6c0c8faa7a1b73b3e23e99878183090231cc8939765d8ed879d038eb6a106a"}
Jul 10 00:20:07.507200 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:07.507161    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:07.607868 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:07.607828    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:07.675877 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:07.675838    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st" event={"ID":"44e54584-138b-4286-a17e-3e16fabe6861","Type":"ContainerStarted","Data":"c480935ac6afc6a2be6e9fdcf517c1b8249beb132a8c426d69da15c52425862f"}
Jul 10 00:20:07.677300 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:07.677273    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf" event={"ID":"a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7","Type":"ContainerStarted","Data":"94f434ef3e642f1a14563dd1ef14e590c9081e235b67c07a53c338d92bb211c5"}
Jul 10 00:20:07.679218 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:07.679146    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:07.712988 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:07.712919    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-8j7st" podStartSLOduration=33.597084475 podStartE2EDuration="35.71289866s" totalImagesPullingTime="2.115814185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:20:07.689998545 +0000 UTC m=+257.392783394" watchObservedRunningTime="2026-07-10 00:20:07.71289866 +0000 UTC m=+257.415683511"
Jul 10 00:20:07.713516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:07.713468    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-blndf" podStartSLOduration=33.510523148 podStartE2EDuration="35.713453485s" totalImagesPullingTime="2.202930337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:20:07.711606122 +0000 UTC m=+257.414390991" watchObservedRunningTime="2026-07-10 00:20:07.713453485 +0000 UTC m=+257.416238334"
Jul 10 00:20:07.821041 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:07.820950    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:07.874291 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:07.874255    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:08.238614 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:08.238580    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:08.541005 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:08.540925    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:20:08.543595 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:08.543564    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed80c3b6-d4f4-490e-a89e-874824a1a195-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zdkv9\" (UID: \"ed80c3b6-d4f4-490e-a89e-874824a1a195\") " pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:20:08.792569 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:08.792483    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2znzl\""
Jul 10 00:20:08.800637 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:08.800615    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zdkv9"
Jul 10 00:20:08.944996 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:08.942787    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zdkv9"]
Jul 10 00:20:09.686781 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:09.686662    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zdkv9" event={"ID":"ed80c3b6-d4f4-490e-a89e-874824a1a195","Type":"ContainerStarted","Data":"be034b83832b88defdb8bc63feb1557a7f0b085707d4503bd8c10987b4f8361d"}
Jul 10 00:20:10.577948 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:10.577900    2559 patch_prober.go:28] interesting pod/image-registry-b8fc76db4-hfjmw container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 10 00:20:10.578140 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:10.577980    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw" podUID="4703c32b-78f0-4cbc-9461-aa323d3fcf67" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 00:20:11.684305 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.684270    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:11.717274 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.716709    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-vrgjp"]
Jul 10 00:20:11.720728 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.720705    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp"
Jul 10 00:20:11.723285 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.723222    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 10 00:20:11.723459 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.723435    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 10 00:20:11.724687 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.724665    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-nrvbf\""
Jul 10 00:20:11.724792 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.724715    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 10 00:20:11.730206 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.730026    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-vrgjp"]
Jul 10 00:20:11.742507 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.742473    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:11.770067 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.770036    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmj7g\" (UniqueName: \"kubernetes.io/projected/4339eff7-c819-4551-94bc-468a6929c710-kube-api-access-cmj7g\") pod \"prometheus-operator-64785b6675-vrgjp\" (UID: \"4339eff7-c819-4551-94bc-468a6929c710\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp"
Jul 10 00:20:11.770241 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.770131    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4339eff7-c819-4551-94bc-468a6929c710-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-vrgjp\" (UID: \"4339eff7-c819-4551-94bc-468a6929c710\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp"
Jul 10 00:20:11.770241 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.770184    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4339eff7-c819-4551-94bc-468a6929c710-metrics-client-ca\") pod \"prometheus-operator-64785b6675-vrgjp\" (UID: \"4339eff7-c819-4551-94bc-468a6929c710\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp"
Jul 10 00:20:11.770241 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.770232    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4339eff7-c819-4551-94bc-468a6929c710-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-vrgjp\" (UID: \"4339eff7-c819-4551-94bc-468a6929c710\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp"
Jul 10 00:20:11.835218 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.835186    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:11.871431 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.871295    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmj7g\" (UniqueName: \"kubernetes.io/projected/4339eff7-c819-4551-94bc-468a6929c710-kube-api-access-cmj7g\") pod \"prometheus-operator-64785b6675-vrgjp\" (UID: \"4339eff7-c819-4551-94bc-468a6929c710\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp"
Jul 10 00:20:11.871431 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.871371    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4339eff7-c819-4551-94bc-468a6929c710-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-vrgjp\" (UID: \"4339eff7-c819-4551-94bc-468a6929c710\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp"
Jul 10 00:20:11.871431 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.871402    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4339eff7-c819-4551-94bc-468a6929c710-metrics-client-ca\") pod \"prometheus-operator-64785b6675-vrgjp\" (UID: \"4339eff7-c819-4551-94bc-468a6929c710\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp"
Jul 10 00:20:11.871750 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.871631    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4339eff7-c819-4551-94bc-468a6929c710-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-vrgjp\" (UID: \"4339eff7-c819-4551-94bc-468a6929c710\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp"
Jul 10 00:20:11.872169 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.872142    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4339eff7-c819-4551-94bc-468a6929c710-metrics-client-ca\") pod \"prometheus-operator-64785b6675-vrgjp\" (UID: \"4339eff7-c819-4551-94bc-468a6929c710\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp"
Jul 10 00:20:11.874450 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.874399    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4339eff7-c819-4551-94bc-468a6929c710-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-vrgjp\" (UID: \"4339eff7-c819-4551-94bc-468a6929c710\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp"
Jul 10 00:20:11.874926 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.874902    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4339eff7-c819-4551-94bc-468a6929c710-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-vrgjp\" (UID: \"4339eff7-c819-4551-94bc-468a6929c710\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp"
Jul 10 00:20:11.879397 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:11.879354    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmj7g\" (UniqueName: \"kubernetes.io/projected/4339eff7-c819-4551-94bc-468a6929c710-kube-api-access-cmj7g\") pod \"prometheus-operator-64785b6675-vrgjp\" (UID: \"4339eff7-c819-4551-94bc-468a6929c710\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp"
Jul 10 00:20:12.034416 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:12.034383    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp"
Jul 10 00:20:12.756748 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:12.756716    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:12.811371 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:12.811327    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:13.179255 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:13.179094    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-vrgjp"]
Jul 10 00:20:13.181657 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:20:13.181628    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4339eff7_c819_4551_94bc_468a6929c710.slice/crio-1ae320be3feb1577f312b00adc2c603dfa21d9b82c8017c4f66d5dbcdec58357 WatchSource:0}: Error finding container 1ae320be3feb1577f312b00adc2c603dfa21d9b82c8017c4f66d5dbcdec58357: Status 404 returned error can't find the container with id 1ae320be3feb1577f312b00adc2c603dfa21d9b82c8017c4f66d5dbcdec58357
Jul 10 00:20:13.706806 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:13.706773    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zdkv9" event={"ID":"ed80c3b6-d4f4-490e-a89e-874824a1a195","Type":"ContainerStarted","Data":"6b761a9015d3cfe116d375664902919d0687e852b71ebc8c86f9ffc9089f2fe3"}
Jul 10 00:20:13.707983 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:13.707954    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp" event={"ID":"4339eff7-c819-4551-94bc-468a6929c710","Type":"ContainerStarted","Data":"1ae320be3feb1577f312b00adc2c603dfa21d9b82c8017c4f66d5dbcdec58357"}
Jul 10 00:20:14.350290 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:14.350215    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:14.412218 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:14.412182    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:14.713559 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:14.713527    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zdkv9" event={"ID":"ed80c3b6-d4f4-490e-a89e-874824a1a195","Type":"ContainerStarted","Data":"f438caaf0fda3d83da71c251523ae8f08e403607660c266d3a6b4b1ef8f1b305"}
Jul 10 00:20:14.985497 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:14.985404    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:15.722167 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:15.722119    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp" event={"ID":"4339eff7-c819-4551-94bc-468a6929c710","Type":"ContainerStarted","Data":"9bc9183f41b44d09f8b6809f2ba28742071a200ed7a731a015d9a5b3e0b9d142"}
Jul 10 00:20:15.722650 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:15.722187    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp" event={"ID":"4339eff7-c819-4551-94bc-468a6929c710","Type":"ContainerStarted","Data":"5c3ac3848c0f968d2e579f199ca6f5daf0b81c3a5dfed3711764872ad94d5d63"}
Jul 10 00:20:15.739217 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:15.739160    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-vrgjp" podStartSLOduration=3.220032429 podStartE2EDuration="4.739140183s" totalImagesPullingTime="1.519107754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:20:15.737243294 +0000 UTC m=+265.440028183" watchObservedRunningTime="2026-07-10 00:20:15.739140183 +0000 UTC m=+265.441925032"
Jul 10 00:20:17.824268 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:17.824233    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:18.038678 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.038634    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:18.079222 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.079141    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wgq7n"]
Jul 10 00:20:18.081970 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.081941    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.086066 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.084887    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 10 00:20:18.086066 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.085010    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 10 00:20:18.086066 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.085472    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 10 00:20:18.086066 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.085860    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dtnhw\""
Jul 10 00:20:18.131293 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.131257    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8faf0e88-2cb7-4551-b15c-bc1c31854b51-node-exporter-textfile\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.131293 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.131303    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8faf0e88-2cb7-4551-b15c-bc1c31854b51-metrics-client-ca\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.131572 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.131349    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8faf0e88-2cb7-4551-b15c-bc1c31854b51-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.131572 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.131386    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8faf0e88-2cb7-4551-b15c-bc1c31854b51-node-exporter-wtmp\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.131572 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.131467    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8faf0e88-2cb7-4551-b15c-bc1c31854b51-sys\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.131572 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.131524    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8faf0e88-2cb7-4551-b15c-bc1c31854b51-node-exporter-accelerators-collector-config\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.131787 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.131575    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8faf0e88-2cb7-4551-b15c-bc1c31854b51-root\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.131787 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.131600    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8faf0e88-2cb7-4551-b15c-bc1c31854b51-node-exporter-tls\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.131787 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.131622    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdf66\" (UniqueName: \"kubernetes.io/projected/8faf0e88-2cb7-4551-b15c-bc1c31854b51-kube-api-access-tdf66\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.138667 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.138640    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:18.199264 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.199229    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:18.232366 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.232337    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8faf0e88-2cb7-4551-b15c-bc1c31854b51-node-exporter-wtmp\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.232543 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.232385    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8faf0e88-2cb7-4551-b15c-bc1c31854b51-sys\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.232543 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.232414    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8faf0e88-2cb7-4551-b15c-bc1c31854b51-node-exporter-accelerators-collector-config\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.232543 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.232440    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8faf0e88-2cb7-4551-b15c-bc1c31854b51-root\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.232543 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.232462    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8faf0e88-2cb7-4551-b15c-bc1c31854b51-node-exporter-tls\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.232543 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.232483    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdf66\" (UniqueName: \"kubernetes.io/projected/8faf0e88-2cb7-4551-b15c-bc1c31854b51-kube-api-access-tdf66\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.232788 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.232538    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8faf0e88-2cb7-4551-b15c-bc1c31854b51-node-exporter-wtmp\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.232788 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.232578    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8faf0e88-2cb7-4551-b15c-bc1c31854b51-node-exporter-textfile\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.232788 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.232603    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8faf0e88-2cb7-4551-b15c-bc1c31854b51-metrics-client-ca\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.232788 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.232611    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8faf0e88-2cb7-4551-b15c-bc1c31854b51-sys\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.232788 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.232629    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8faf0e88-2cb7-4551-b15c-bc1c31854b51-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.233024 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.232908    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8faf0e88-2cb7-4551-b15c-bc1c31854b51-node-exporter-textfile\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.233024 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.232975    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8faf0e88-2cb7-4551-b15c-bc1c31854b51-root\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.233173 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.233151    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8faf0e88-2cb7-4551-b15c-bc1c31854b51-node-exporter-accelerators-collector-config\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.233510 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.233473    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8faf0e88-2cb7-4551-b15c-bc1c31854b51-metrics-client-ca\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.235309 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.235209    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8faf0e88-2cb7-4551-b15c-bc1c31854b51-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.235444 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.235355    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8faf0e88-2cb7-4551-b15c-bc1c31854b51-node-exporter-tls\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.243612 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.243586    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdf66\" (UniqueName: \"kubernetes.io/projected/8faf0e88-2cb7-4551-b15c-bc1c31854b51-kube-api-access-tdf66\") pod \"node-exporter-wgq7n\" (UID: \"8faf0e88-2cb7-4551-b15c-bc1c31854b51\") " pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.302491 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.302445    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:18.373870 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.373792    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:18.394004 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.393967    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wgq7n"
Jul 10 00:20:18.730123 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.730089    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:18.865228 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:18.865197    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:19.053429 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:19.053308    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:19.118305 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:19.118275    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:19.265233 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:19.265188    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:20.575837 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:20.575798    2559 patch_prober.go:28] interesting pod/image-registry-b8fc76db4-hfjmw container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 10 00:20:20.576251 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:20.575862    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw" podUID="4703c32b-78f0-4cbc-9461-aa323d3fcf67" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 00:20:20.988364 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:20.988251    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:21.059292 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:21.059259    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:21.294046 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:21.293932    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:21.494333 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:21.494269    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:22.858657 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:22.858600    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:22.862282 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:22.862255    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-r7btx"]
Jul 10 00:20:22.865595 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:22.865573    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-r7btx"
Jul 10 00:20:22.868190 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:22.868169    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-csgd8\""
Jul 10 00:20:22.868190 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:22.868180    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 10 00:20:22.875946 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:22.875920    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-r7btx"]
Jul 10 00:20:22.914674 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:22.914641    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:22.975846 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:22.975805    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/724ba4e9-fa99-42ac-afe2-b8044d78d131-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-r7btx\" (UID: \"724ba4e9-fa99-42ac-afe2-b8044d78d131\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-r7btx"
Jul 10 00:20:22.976016 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:22.975911    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cdw8\" (UniqueName: \"kubernetes.io/projected/724ba4e9-fa99-42ac-afe2-b8044d78d131-kube-api-access-7cdw8\") pod \"monitoring-plugin-5f68df7f58-r7btx\" (UID: \"724ba4e9-fa99-42ac-afe2-b8044d78d131\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-r7btx"
Jul 10 00:20:23.014367 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:23.014334    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:23.076506 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:23.076478    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/724ba4e9-fa99-42ac-afe2-b8044d78d131-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-r7btx\" (UID: \"724ba4e9-fa99-42ac-afe2-b8044d78d131\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-r7btx"
Jul 10 00:20:23.076663 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:23.076537    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cdw8\" (UniqueName: \"kubernetes.io/projected/724ba4e9-fa99-42ac-afe2-b8044d78d131-kube-api-access-7cdw8\") pod \"monitoring-plugin-5f68df7f58-r7btx\" (UID: \"724ba4e9-fa99-42ac-afe2-b8044d78d131\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-r7btx"
Jul 10 00:20:23.078974 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:23.078946    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/724ba4e9-fa99-42ac-afe2-b8044d78d131-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-r7btx\" (UID: \"724ba4e9-fa99-42ac-afe2-b8044d78d131\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-r7btx"
Jul 10 00:20:23.084805 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:23.084782    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cdw8\" (UniqueName: \"kubernetes.io/projected/724ba4e9-fa99-42ac-afe2-b8044d78d131-kube-api-access-7cdw8\") pod \"monitoring-plugin-5f68df7f58-r7btx\" (UID: \"724ba4e9-fa99-42ac-afe2-b8044d78d131\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-r7btx"
Jul 10 00:20:23.180284 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:23.180195    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-r7btx"
Jul 10 00:20:23.860995 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:20:23.860959    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8faf0e88_2cb7_4551_b15c_bc1c31854b51.slice/crio-d1bc2957db0c3566066b09c97a336518223506a64d46b411ea4a577d018cfed8 WatchSource:0}: Error finding container d1bc2957db0c3566066b09c97a336518223506a64d46b411ea4a577d018cfed8: Status 404 returned error can't find the container with id d1bc2957db0c3566066b09c97a336518223506a64d46b411ea4a577d018cfed8
Jul 10 00:20:23.986827 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:23.986799    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-r7btx"]
Jul 10 00:20:23.989254 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:20:23.989221    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod724ba4e9_fa99_42ac_afe2_b8044d78d131.slice/crio-81aec9c5576383c1e1b8371cf63869a331fa8547f73ed04de42fd0e9ace8f2b5 WatchSource:0}: Error finding container 81aec9c5576383c1e1b8371cf63869a331fa8547f73ed04de42fd0e9ace8f2b5: Status 404 returned error can't find the container with id 81aec9c5576383c1e1b8371cf63869a331fa8547f73ed04de42fd0e9ace8f2b5
Jul 10 00:20:23.993745 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:23.993716    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:24.095664 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.095631    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:24.199409 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.199375    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b78844f9-ft8jm"]
Jul 10 00:20:24.202626 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.202597    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.206007 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.205364    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 10 00:20:24.206007 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.205615    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 10 00:20:24.206007 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.205823    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 10 00:20:24.206893 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.206670    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 10 00:20:24.207499 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.207300    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 10 00:20:24.207684 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.207656    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-g7w2x\""
Jul 10 00:20:24.215184 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.212862    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 10 00:20:24.215395 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.215027    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b78844f9-ft8jm"]
Jul 10 00:20:24.277852 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.275004    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:24.279039 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.279010    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 00:20:24.284716 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.284688    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.291625 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.290472    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb950baa-e857-4d57-a325-00afc9b747c4-console-oauth-config\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.291625 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.290516    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwxmq\" (UniqueName: \"kubernetes.io/projected/eb950baa-e857-4d57-a325-00afc9b747c4-kube-api-access-nwxmq\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.291625 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.290572    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-console-config\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.291625 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.290597    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-oauth-serving-cert\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.291625 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.290697    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-trusted-ca-bundle\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.291625 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.290726    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-service-ca\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.291625 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.290759    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb950baa-e857-4d57-a325-00afc9b747c4-console-serving-cert\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.291625 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.290981    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 10 00:20:24.291625 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.291022    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 10 00:20:24.291625 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.291032    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 10 00:20:24.291625 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.290983    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 10 00:20:24.291625 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.291220    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 10 00:20:24.291625 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.291337    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 10 00:20:24.291625 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.291423    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 10 00:20:24.292540 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.291736    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 10 00:20:24.292540 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.291940    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 10 00:20:24.292540 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.292122    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 10 00:20:24.292877 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.292861    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 10 00:20:24.293869 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.293845    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 00:20:24.294054 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.293946    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zc5b7\""
Jul 10 00:20:24.294194 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.294174    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-41uo784l26j6d\""
Jul 10 00:20:24.294280 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.294125    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 10 00:20:24.294421 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.294078    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 10 00:20:24.294596 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.294470    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 10 00:20:24.296158 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.296137    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 10 00:20:24.392197 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.391688    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-service-ca\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.392197 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.391738    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb950baa-e857-4d57-a325-00afc9b747c4-console-serving-cert\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.392197 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.391772    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.392197 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.391802    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.392197 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.391843    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.392197 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.391868    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.392197 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.391893    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.392197 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.391925    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.392197 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.391950    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.392197 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.391976    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4xfn\" (UniqueName: \"kubernetes.io/projected/3937d9c7-89ab-4972-b3af-f40c98f3274c-kube-api-access-z4xfn\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.392197 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392007    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb950baa-e857-4d57-a325-00afc9b747c4-console-oauth-config\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.392197 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392039    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwxmq\" (UniqueName: \"kubernetes.io/projected/eb950baa-e857-4d57-a325-00afc9b747c4-kube-api-access-nwxmq\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.392197 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392176    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.393012 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392217    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.393012 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392247    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-console-config\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.393012 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392273    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-oauth-serving-cert\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.393012 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392329    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.393012 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392355    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-config\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.393012 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392373    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.393012 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392392    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3937d9c7-89ab-4972-b3af-f40c98f3274c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.393012 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392406    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-web-config\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.393012 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392471    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.393012 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392496    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.393012 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392538    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3937d9c7-89ab-4972-b3af-f40c98f3274c-config-out\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.393012 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392564    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-trusted-ca-bundle\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.393012 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392590    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.393012 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.392618    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.394839 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.394816    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-service-ca\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.394996 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.394925    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-oauth-serving-cert\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.396077 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.395609    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-trusted-ca-bundle\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.396403 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.396364    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-console-config\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.399259 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.399212    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb950baa-e857-4d57-a325-00afc9b747c4-console-oauth-config\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.399857 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.399834    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb950baa-e857-4d57-a325-00afc9b747c4-console-serving-cert\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.401817 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.401774    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwxmq\" (UniqueName: \"kubernetes.io/projected/eb950baa-e857-4d57-a325-00afc9b747c4-kube-api-access-nwxmq\") pod \"console-6b78844f9-ft8jm\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") " pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.494075 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.493974    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494075 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494024    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494075 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494065    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494377 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494094    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-config\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494377 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494117    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494377 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494145    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3937d9c7-89ab-4972-b3af-f40c98f3274c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494377 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494178    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-web-config\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494377 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494218    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494377 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494243    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494377 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494293    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3937d9c7-89ab-4972-b3af-f40c98f3274c-config-out\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494377 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494338    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494377 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494366    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494795 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494402    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494795 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494428    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494795 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494471    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494795 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494494    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494795 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494517    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494795 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494550    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494795 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494573    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.494795 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.494606    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4xfn\" (UniqueName: \"kubernetes.io/projected/3937d9c7-89ab-4972-b3af-f40c98f3274c-kube-api-access-z4xfn\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.496053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.495703    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.496053 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.495771    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.505132 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.497031    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.505132 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.499230    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.505132 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.499726    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.505132 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.500237    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.505132 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.501548    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.505132 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.502442    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.505132 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.502706    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.507688 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.507629    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4xfn\" (UniqueName: \"kubernetes.io/projected/3937d9c7-89ab-4972-b3af-f40c98f3274c-kube-api-access-z4xfn\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.508240 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.508190    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.508437 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.508249    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-web-config\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.511909 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.510362    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.512427 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.512380    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-config\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.514604 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.514552    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3937d9c7-89ab-4972-b3af-f40c98f3274c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.520087 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.519065    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.520087 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.519266    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3937d9c7-89ab-4972-b3af-f40c98f3274c-config-out\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.520087 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.519381    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.520087 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.519462    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:24.520087 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.520084    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.520983 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.520935    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.607942 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.607597    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:24.751719 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.751271    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b78844f9-ft8jm"]
Jul 10 00:20:24.755808 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:20:24.755771    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb950baa_e857_4d57_a325_00afc9b747c4.slice/crio-69a76bc2ad6fc7389a1c4604879ee4c642fc3ec9d31a2f15b82d5d0954809746 WatchSource:0}: Error finding container 69a76bc2ad6fc7389a1c4604879ee4c642fc3ec9d31a2f15b82d5d0954809746: Status 404 returned error can't find the container with id 69a76bc2ad6fc7389a1c4604879ee4c642fc3ec9d31a2f15b82d5d0954809746
Jul 10 00:20:24.759411 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.759339    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-r7btx" event={"ID":"724ba4e9-fa99-42ac-afe2-b8044d78d131","Type":"ContainerStarted","Data":"81aec9c5576383c1e1b8371cf63869a331fa8547f73ed04de42fd0e9ace8f2b5"}
Jul 10 00:20:24.776298 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.775464    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-cgdmm" event={"ID":"0f28bafe-bd50-4228-aee1-04e545bbf94a","Type":"ContainerStarted","Data":"4a20e5139e802435112d3661345fcf40ecfd62e38c419c03cacc78571c017cab"}
Jul 10 00:20:24.778357 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.778269    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-cgdmm"
Jul 10 00:20:24.781005 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.780959    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-cgdmm"
Jul 10 00:20:24.785817 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.785079    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zdkv9" event={"ID":"ed80c3b6-d4f4-490e-a89e-874824a1a195","Type":"ContainerStarted","Data":"c0f79f9d9b910f9b2ad2717c8afeb4d318e2412627cd23f248f77d984198a91e"}
Jul 10 00:20:24.787555 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.787485    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wgq7n" event={"ID":"8faf0e88-2cb7-4551-b15c-bc1c31854b51","Type":"ContainerStarted","Data":"d1bc2957db0c3566066b09c97a336518223506a64d46b411ea4a577d018cfed8"}
Jul 10 00:20:24.822417 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.822186    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zdkv9" podStartSLOduration=34.044155957 podStartE2EDuration="48.822163809s" totalImagesPullingTime="14.778007852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:19:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:20:24.821514264 +0000 UTC m=+274.524299139" watchObservedRunningTime="2026-07-10 00:20:24.822163809 +0000 UTC m=+274.524948659"
Jul 10 00:20:24.823519 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.823455    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-cgdmm" podStartSLOduration=1.969744515 podStartE2EDuration="24.823438035s" totalImagesPullingTime="22.85369352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:20:24.795269838 +0000 UTC m=+274.498054687" watchObservedRunningTime="2026-07-10 00:20:24.823438035 +0000 UTC m=+274.526222883"
Jul 10 00:20:24.845523 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:24.845405    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 00:20:25.314095 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:20:25.314056    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3937d9c7_89ab_4972_b3af_f40c98f3274c.slice/crio-54d6ce8a51f54f8e22dd0e71d80607820b6b4caa74576b7e65035d654f5ba09f WatchSource:0}: Error finding container 54d6ce8a51f54f8e22dd0e71d80607820b6b4caa74576b7e65035d654f5ba09f: Status 404 returned error can't find the container with id 54d6ce8a51f54f8e22dd0e71d80607820b6b4caa74576b7e65035d654f5ba09f
Jul 10 00:20:25.590768 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:25.590685    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw" podUID="4703c32b-78f0-4cbc-9461-aa323d3fcf67" containerName="registry" containerID="cri-o://f12243e2597edbbb22bde40a6ca51baed98a98cccbde274a01c3e148d6d85997" gracePeriod=30
Jul 10 00:20:25.796521 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:25.796397    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3937d9c7-89ab-4972-b3af-f40c98f3274c","Type":"ContainerStarted","Data":"54d6ce8a51f54f8e22dd0e71d80607820b6b4caa74576b7e65035d654f5ba09f"}
Jul 10 00:20:25.801075 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:25.800392    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b78844f9-ft8jm" event={"ID":"eb950baa-e857-4d57-a325-00afc9b747c4","Type":"ContainerStarted","Data":"69a76bc2ad6fc7389a1c4604879ee4c642fc3ec9d31a2f15b82d5d0954809746"}
Jul 10 00:20:26.525139 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.524967    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:20:26.627763 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.627018    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4703c32b-78f0-4cbc-9461-aa323d3fcf67-trusted-ca\") pod \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") "
Jul 10 00:20:26.627763 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.627064    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls\") pod \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") "
Jul 10 00:20:26.627763 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.627113    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4703c32b-78f0-4cbc-9461-aa323d3fcf67-installation-pull-secrets\") pod \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") "
Jul 10 00:20:26.627763 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.627172    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2c9v\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-kube-api-access-q2c9v\") pod \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") "
Jul 10 00:20:26.627763 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.627206    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4703c32b-78f0-4cbc-9461-aa323d3fcf67-ca-trust-extracted\") pod \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") "
Jul 10 00:20:26.627763 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.627247    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4703c32b-78f0-4cbc-9461-aa323d3fcf67-image-registry-private-configuration\") pod \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") "
Jul 10 00:20:26.627763 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.627300    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-certificates\") pod \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") "
Jul 10 00:20:26.627763 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.627345    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-bound-sa-token\") pod \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\" (UID: \"4703c32b-78f0-4cbc-9461-aa323d3fcf67\") "
Jul 10 00:20:26.630197 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.629488    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4703c32b-78f0-4cbc-9461-aa323d3fcf67-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4703c32b-78f0-4cbc-9461-aa323d3fcf67" (UID: "4703c32b-78f0-4cbc-9461-aa323d3fcf67"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:20:26.630197 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.629553    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4703c32b-78f0-4cbc-9461-aa323d3fcf67" (UID: "4703c32b-78f0-4cbc-9461-aa323d3fcf67"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:20:26.631596 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.631549    2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-certificates\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:20:26.631596 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.631576    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4703c32b-78f0-4cbc-9461-aa323d3fcf67-trusted-ca\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:20:26.632376 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.632307    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4703c32b-78f0-4cbc-9461-aa323d3fcf67-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4703c32b-78f0-4cbc-9461-aa323d3fcf67" (UID: "4703c32b-78f0-4cbc-9461-aa323d3fcf67"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:20:26.636460 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.636410    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-kube-api-access-q2c9v" (OuterVolumeSpecName: "kube-api-access-q2c9v") pod "4703c32b-78f0-4cbc-9461-aa323d3fcf67" (UID: "4703c32b-78f0-4cbc-9461-aa323d3fcf67"). InnerVolumeSpecName "kube-api-access-q2c9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:20:26.636954 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.636918    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4703c32b-78f0-4cbc-9461-aa323d3fcf67" (UID: "4703c32b-78f0-4cbc-9461-aa323d3fcf67"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:20:26.637397 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.637362    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4703c32b-78f0-4cbc-9461-aa323d3fcf67" (UID: "4703c32b-78f0-4cbc-9461-aa323d3fcf67"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:20:26.642429 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.642036    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4703c32b-78f0-4cbc-9461-aa323d3fcf67-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4703c32b-78f0-4cbc-9461-aa323d3fcf67" (UID: "4703c32b-78f0-4cbc-9461-aa323d3fcf67"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 00:20:26.642705 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.642521    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4703c32b-78f0-4cbc-9461-aa323d3fcf67-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4703c32b-78f0-4cbc-9461-aa323d3fcf67" (UID: "4703c32b-78f0-4cbc-9461-aa323d3fcf67"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:20:26.732326 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.732252    2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4703c32b-78f0-4cbc-9461-aa323d3fcf67-image-registry-private-configuration\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:20:26.732326 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.732290    2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-bound-sa-token\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:20:26.732326 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.732307    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-registry-tls\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:20:26.732619 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.732345    2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4703c32b-78f0-4cbc-9461-aa323d3fcf67-installation-pull-secrets\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:20:26.732619 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.732361    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2c9v\" (UniqueName: \"kubernetes.io/projected/4703c32b-78f0-4cbc-9461-aa323d3fcf67-kube-api-access-q2c9v\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:20:26.732619 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.732375    2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4703c32b-78f0-4cbc-9461-aa323d3fcf67-ca-trust-extracted\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:20:26.807431 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.807050    2559 generic.go:356] "Generic (PLEG): container finished" podID="4703c32b-78f0-4cbc-9461-aa323d3fcf67" containerID="f12243e2597edbbb22bde40a6ca51baed98a98cccbde274a01c3e148d6d85997" exitCode=0
Jul 10 00:20:26.807431 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.807165    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw" event={"ID":"4703c32b-78f0-4cbc-9461-aa323d3fcf67","Type":"ContainerDied","Data":"f12243e2597edbbb22bde40a6ca51baed98a98cccbde274a01c3e148d6d85997"}
Jul 10 00:20:26.807431 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.807223    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw"
Jul 10 00:20:26.807431 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.807255    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b8fc76db4-hfjmw" event={"ID":"4703c32b-78f0-4cbc-9461-aa323d3fcf67","Type":"ContainerDied","Data":"052f50b9c08ad59e292507da4be48d7fde5e55ab9c4a5b8d86215bbcb709e069"}
Jul 10 00:20:26.807431 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.807288    2559 scope.go:117] "RemoveContainer" containerID="f12243e2597edbbb22bde40a6ca51baed98a98cccbde274a01c3e148d6d85997"
Jul 10 00:20:26.809738 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.809701    2559 generic.go:356] "Generic (PLEG): container finished" podID="8faf0e88-2cb7-4551-b15c-bc1c31854b51" containerID="7eef011abeecb933ceb8146db7192031169cf16721e0890e0d90e85d4c87afd4" exitCode=0
Jul 10 00:20:26.809911 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.809824    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wgq7n" event={"ID":"8faf0e88-2cb7-4551-b15c-bc1c31854b51","Type":"ContainerDied","Data":"7eef011abeecb933ceb8146db7192031169cf16721e0890e0d90e85d4c87afd4"}
Jul 10 00:20:26.815948 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.815896    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-r7btx" event={"ID":"724ba4e9-fa99-42ac-afe2-b8044d78d131","Type":"ContainerStarted","Data":"d2e9f31522231844e466c9342fa062b4ff0e95f34375a9216bf8a7bdadc76770"}
Jul 10 00:20:26.816063 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.815949    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-r7btx"
Jul 10 00:20:26.823198 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.823178    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-r7btx"
Jul 10 00:20:26.826585 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.826566    2559 scope.go:117] "RemoveContainer" containerID="f12243e2597edbbb22bde40a6ca51baed98a98cccbde274a01c3e148d6d85997"
Jul 10 00:20:26.827774 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:20:26.827447    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f12243e2597edbbb22bde40a6ca51baed98a98cccbde274a01c3e148d6d85997\": container with ID starting with f12243e2597edbbb22bde40a6ca51baed98a98cccbde274a01c3e148d6d85997 not found: ID does not exist" containerID="f12243e2597edbbb22bde40a6ca51baed98a98cccbde274a01c3e148d6d85997"
Jul 10 00:20:26.827774 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.827493    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f12243e2597edbbb22bde40a6ca51baed98a98cccbde274a01c3e148d6d85997"} err="failed to get container status \"f12243e2597edbbb22bde40a6ca51baed98a98cccbde274a01c3e148d6d85997\": rpc error: code = NotFound desc = could not find container \"f12243e2597edbbb22bde40a6ca51baed98a98cccbde274a01c3e148d6d85997\": container with ID starting with f12243e2597edbbb22bde40a6ca51baed98a98cccbde274a01c3e148d6d85997 not found: ID does not exist"
Jul 10 00:20:26.845853 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.845824    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b8fc76db4-hfjmw"]
Jul 10 00:20:26.847531 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.847510    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-b8fc76db4-hfjmw"]
Jul 10 00:20:26.861685 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:26.861087    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-r7btx" podStartSLOduration=2.344580551 podStartE2EDuration="4.861066871s" totalImagesPullingTime="2.51648632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:20:26.860825 +0000 UTC m=+276.563609873" watchObservedRunningTime="2026-07-10 00:20:26.861066871 +0000 UTC m=+276.563851733"
Jul 10 00:20:27.824796 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:27.824761    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wgq7n" event={"ID":"8faf0e88-2cb7-4551-b15c-bc1c31854b51","Type":"ContainerStarted","Data":"ca72baa04964e153646aa44189ea963565347729eaac7ca5e7a29eba9968e5c4"}
Jul 10 00:20:27.825360 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:27.824817    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wgq7n" event={"ID":"8faf0e88-2cb7-4551-b15c-bc1c31854b51","Type":"ContainerStarted","Data":"25a9aa3b2f50693cbe56a9a0ae2ec28258adfe3c2b19f098ff6a6f5683ea23cd"}
Jul 10 00:20:27.845637 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:27.844729    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wgq7n" podStartSLOduration=7.383464758 podStartE2EDuration="9.844708008s" totalImagesPullingTime="2.369709279s" totalInitContainerRuntime="91.533971ms" isStatefulPod=true podCreationTimestamp="2026-07-10 00:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:20:27.843074669 +0000 UTC m=+277.545859530" watchObservedRunningTime="2026-07-10 00:20:27.844708008 +0000 UTC m=+277.547492861"
Jul 10 00:20:28.839215 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:28.839105    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4703c32b-78f0-4cbc-9461-aa323d3fcf67" path="/var/lib/kubelet/pods/4703c32b-78f0-4cbc-9461-aa323d3fcf67/volumes"
Jul 10 00:20:29.834183 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:29.834133    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b78844f9-ft8jm" event={"ID":"eb950baa-e857-4d57-a325-00afc9b747c4","Type":"ContainerStarted","Data":"17a255fcc36bd4446c703a807c362b9c8bb39f5e7cdb4449aebaab33dd20f7a0"}
Jul 10 00:20:29.835882 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:29.835848    2559 generic.go:356] "Generic (PLEG): container finished" podID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerID="5f5d57eec3459e97cf1c8b700f47211aa8c5b6080339ec12e87aee7e328cbbc4" exitCode=0
Jul 10 00:20:29.836016 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:29.835916    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3937d9c7-89ab-4972-b3af-f40c98f3274c","Type":"ContainerDied","Data":"5f5d57eec3459e97cf1c8b700f47211aa8c5b6080339ec12e87aee7e328cbbc4"}
Jul 10 00:20:29.853536 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:29.853478    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b78844f9-ft8jm" podStartSLOduration=1.221311369 podStartE2EDuration="5.853462345s" totalImagesPullingTime="4.632150976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:20:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:20:29.852137903 +0000 UTC m=+279.554922754" watchObservedRunningTime="2026-07-10 00:20:29.853462345 +0000 UTC m=+279.556247244"
Jul 10 00:20:33.383847 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:20:33.383797    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-j8x4j" podUID="6e7a4774-431a-40c0-8c44-f3f929be8e0f"
Jul 10 00:20:33.492221 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:33.492178    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:33.545844 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:33.545802    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:33.858696 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:33.858662    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:20:34.520112 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:34.520076    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:34.520618 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:34.520173    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:34.521496 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:34.521474    2559 patch_prober.go:28] interesting pod/console-6b78844f9-ft8jm container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.21:8443/health\": dial tcp 10.134.0.21:8443: connect: connection refused" start-of-body=
Jul 10 00:20:34.521582 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:34.521520    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6b78844f9-ft8jm" podUID="eb950baa-e857-4d57-a325-00afc9b747c4" containerName="console" probeResult="failure" output="Get \"https://10.134.0.21:8443/health\": dial tcp 10.134.0.21:8443: connect: connection refused"
Jul 10 00:20:34.868909 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:34.868870    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3937d9c7-89ab-4972-b3af-f40c98f3274c","Type":"ContainerStarted","Data":"886c6cb2fe6f36dddd14f93d51278d52d903eb0c79c42a375eed110a993bee6f"}
Jul 10 00:20:34.869033 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:34.868965    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3937d9c7-89ab-4972-b3af-f40c98f3274c","Type":"ContainerStarted","Data":"f446c37f66beddbebd0524637f7dd7c2e9b6fade4f0766c5028b01ad8eda689c"}
Jul 10 00:20:36.101481 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:36.101439    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:36.202278 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:36.202242    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:37.137077 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:37.137037    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:20:37.139544 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:37.139520    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e7a4774-431a-40c0-8c44-f3f929be8e0f-metrics-tls\") pod \"dns-default-j8x4j\" (UID: \"6e7a4774-431a-40c0-8c44-f3f929be8e0f\") " pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:20:37.161939 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:37.161915    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-crl8x\""
Jul 10 00:20:37.170100 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:37.170076    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:20:37.337674 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:37.337646    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j8x4j"]
Jul 10 00:20:37.340979 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:20:37.340950    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e7a4774_431a_40c0_8c44_f3f929be8e0f.slice/crio-60830f296e19bca4c847bd64c177a36b12196d064647613a5aa6b4f26fd6d3b5 WatchSource:0}: Error finding container 60830f296e19bca4c847bd64c177a36b12196d064647613a5aa6b4f26fd6d3b5: Status 404 returned error can't find the container with id 60830f296e19bca4c847bd64c177a36b12196d064647613a5aa6b4f26fd6d3b5
Jul 10 00:20:37.882027 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:37.881981    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j8x4j" event={"ID":"6e7a4774-431a-40c0-8c44-f3f929be8e0f","Type":"ContainerStarted","Data":"60830f296e19bca4c847bd64c177a36b12196d064647613a5aa6b4f26fd6d3b5"}
Jul 10 00:20:38.890080 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:38.890043    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3937d9c7-89ab-4972-b3af-f40c98f3274c","Type":"ContainerStarted","Data":"2f964d30ac92fa090acdb7922d5bed6f10ad8ac2534a8b01050a8cf39013b516"}
Jul 10 00:20:38.890556 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:38.890125    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3937d9c7-89ab-4972-b3af-f40c98f3274c","Type":"ContainerStarted","Data":"ba6618e687500e09288705c60972d790c9a64447cd54801264cf62e158e8c2f9"}
Jul 10 00:20:38.890556 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:38.890169    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3937d9c7-89ab-4972-b3af-f40c98f3274c","Type":"ContainerStarted","Data":"1cb59a65402f1465160fe8ad92706145c57a43f3bfdf6092880f0fc17a5c041d"}
Jul 10 00:20:38.890556 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:38.890216    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3937d9c7-89ab-4972-b3af-f40c98f3274c","Type":"ContainerStarted","Data":"79086510fea92853e03fff9a89e8a2ea1a4b315c2131c4307c4e0f5df63364c1"}
Jul 10 00:20:38.920274 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:38.920170    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.527303409 podStartE2EDuration="14.920152382s" totalImagesPullingTime="12.360527999s" totalInitContainerRuntime="32.320974ms" isStatefulPod=false podCreationTimestamp="2026-07-10 00:20:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:20:38.918982612 +0000 UTC m=+288.621767522" watchObservedRunningTime="2026-07-10 00:20:38.920152382 +0000 UTC m=+288.622937229"
Jul 10 00:20:39.608746 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:39.608711    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:20:40.901764 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:40.901675    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j8x4j" event={"ID":"6e7a4774-431a-40c0-8c44-f3f929be8e0f","Type":"ContainerStarted","Data":"8e137115f54dc4a7d4152135a3a9f9cded98458d33c2afb02deb4b492b9b550f"}
Jul 10 00:20:40.901764 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:40.901734    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j8x4j" event={"ID":"6e7a4774-431a-40c0-8c44-f3f929be8e0f","Type":"ContainerStarted","Data":"0b8b4c7f1250b1b2d8cac26804f13c9534796da09b7df028e2aa6b3535d42eee"}
Jul 10 00:20:40.901764 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:40.901763    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:20:40.917083 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:40.917031    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-j8x4j" podStartSLOduration=250.911101977 podStartE2EDuration="4m13.917013047s" totalImagesPullingTime="3.00591107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:20:40.916047375 +0000 UTC m=+290.618832224" watchObservedRunningTime="2026-07-10 00:20:40.917013047 +0000 UTC m=+290.619797894"
Jul 10 00:20:42.910852 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:42.910769    2559 generic.go:356] "Generic (PLEG): container finished" podID="0eef5919-029b-4087-93e6-c44b8fdce6fa" containerID="d5b1232befe6f06c84a65953bceda7a9886a4f0152b907c116e599ae19fe3798" exitCode=0
Jul 10 00:20:42.910852 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:42.910831    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82" event={"ID":"0eef5919-029b-4087-93e6-c44b8fdce6fa","Type":"ContainerDied","Data":"d5b1232befe6f06c84a65953bceda7a9886a4f0152b907c116e599ae19fe3798"}
Jul 10 00:20:42.911258 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:42.911127    2559 scope.go:117] "RemoveContainer" containerID="d5b1232befe6f06c84a65953bceda7a9886a4f0152b907c116e599ae19fe3798"
Jul 10 00:20:42.928884 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:42.928849    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:43.916387 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:43.916354    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-tsm82" event={"ID":"0eef5919-029b-4087-93e6-c44b8fdce6fa","Type":"ContainerStarted","Data":"143ef260f6bace838b68b5c3e449e9cdfeeb4121ff616ace9775f2d6b15eb9b9"}
Jul 10 00:20:44.420662 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:44.420629    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:44.524398 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:44.524366    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:44.528368 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:44.528346    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:20:44.685168 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:44.685084    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:44.735218 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:44.735188    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:45.107185 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:45.107154    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:45.168724 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:45.168691    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:45.879406 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:45.879372    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:45.931101 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:45.931065    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:46.821436 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:46.821398    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:46.922219 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:46.922187    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:47.608717 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:47.608679    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b78844f9-ft8jm"]
Jul 10 00:20:47.708379 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:47.708346    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:47.776642 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:47.776608    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:47.829705 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:47.829657    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:50.695992 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:50.695967    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:20:50.696641 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:50.696620    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:20:50.700464 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:50.700441    2559 kubelet.go:1648] "Image garbage collection succeeded"
Jul 10 00:20:50.908244 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:50.908100    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-j8x4j"
Jul 10 00:20:51.120168 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:51.029445    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:51.120168 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:51.078728    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:51.233186 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:51.233156    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:51.285084 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:51.285055    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:54.059005 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:54.058968    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:20:54.110491 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:20:54.110457    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:00.437117 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:00.437081    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:00.539843 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:00.539810    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:03.132739 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:03.132697    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:03.189658 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:03.189598    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:04.827998 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:04.827951    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:04.877026 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:04.876990    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:06.931032 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:06.930987    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:07.031511 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:07.031475    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:07.475038 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:07.475002    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:07.554695 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:07.554660    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:12.035747 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:12.035713    2559 generic.go:356] "Generic (PLEG): container finished" podID="db79ea28-a194-4f03-ab3c-a6faee9a011d" containerID="f6a5d404fe83ad496864c5e4c08f87ea69196ea3451af594ea2233d8c9e21000" exitCode=0
Jul 10 00:21:12.036116 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:12.035786    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2" event={"ID":"db79ea28-a194-4f03-ab3c-a6faee9a011d","Type":"ContainerDied","Data":"f6a5d404fe83ad496864c5e4c08f87ea69196ea3451af594ea2233d8c9e21000"}
Jul 10 00:21:12.036156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:12.036120    2559 scope.go:117] "RemoveContainer" containerID="f6a5d404fe83ad496864c5e4c08f87ea69196ea3451af594ea2233d8c9e21000"
Jul 10 00:21:12.615475 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:12.615338    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:12.632919 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:12.632875    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b78844f9-ft8jm" podUID="eb950baa-e857-4d57-a325-00afc9b747c4" containerName="console" containerID="cri-o://17a255fcc36bd4446c703a807c362b9c8bb39f5e7cdb4449aebaab33dd20f7a0" gracePeriod=15
Jul 10 00:21:12.909983 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:12.909949    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:21:13.041911 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.041872    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-dnvt2" event={"ID":"db79ea28-a194-4f03-ab3c-a6faee9a011d","Type":"ContainerStarted","Data":"87e9db0dac798410af9166ad2d2c8ef880f11afc75dfb923c71e976cd63acb7e"}
Jul 10 00:21:13.043115 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.043090    2559 generic.go:356] "Generic (PLEG): container finished" podID="eb950baa-e857-4d57-a325-00afc9b747c4" containerID="17a255fcc36bd4446c703a807c362b9c8bb39f5e7cdb4449aebaab33dd20f7a0" exitCode=0
Jul 10 00:21:13.043241 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.043122    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b78844f9-ft8jm" event={"ID":"eb950baa-e857-4d57-a325-00afc9b747c4","Type":"ContainerDied","Data":"17a255fcc36bd4446c703a807c362b9c8bb39f5e7cdb4449aebaab33dd20f7a0"}
Jul 10 00:21:13.043241 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.043167    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b78844f9-ft8jm" event={"ID":"eb950baa-e857-4d57-a325-00afc9b747c4","Type":"ContainerDied","Data":"69a76bc2ad6fc7389a1c4604879ee4c642fc3ec9d31a2f15b82d5d0954809746"}
Jul 10 00:21:13.043241 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.043186    2559 scope.go:117] "RemoveContainer" containerID="17a255fcc36bd4446c703a807c362b9c8bb39f5e7cdb4449aebaab33dd20f7a0"
Jul 10 00:21:13.043241 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.043130    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b78844f9-ft8jm"
Jul 10 00:21:13.048059 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.048041    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb950baa-e857-4d57-a325-00afc9b747c4-console-serving-cert\") pod \"eb950baa-e857-4d57-a325-00afc9b747c4\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") "
Jul 10 00:21:13.048155 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.048100    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-service-ca\") pod \"eb950baa-e857-4d57-a325-00afc9b747c4\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") "
Jul 10 00:21:13.048155 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.048125    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-console-config\") pod \"eb950baa-e857-4d57-a325-00afc9b747c4\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") "
Jul 10 00:21:13.048238 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.048221    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-oauth-serving-cert\") pod \"eb950baa-e857-4d57-a325-00afc9b747c4\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") "
Jul 10 00:21:13.048372 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.048353    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb950baa-e857-4d57-a325-00afc9b747c4-console-oauth-config\") pod \"eb950baa-e857-4d57-a325-00afc9b747c4\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") "
Jul 10 00:21:13.048457 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.048386    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwxmq\" (UniqueName: \"kubernetes.io/projected/eb950baa-e857-4d57-a325-00afc9b747c4-kube-api-access-nwxmq\") pod \"eb950baa-e857-4d57-a325-00afc9b747c4\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") "
Jul 10 00:21:13.048457 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.048425    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-trusted-ca-bundle\") pod \"eb950baa-e857-4d57-a325-00afc9b747c4\" (UID: \"eb950baa-e857-4d57-a325-00afc9b747c4\") "
Jul 10 00:21:13.048851 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.048569    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-console-config" (OuterVolumeSpecName: "console-config") pod "eb950baa-e857-4d57-a325-00afc9b747c4" (UID: "eb950baa-e857-4d57-a325-00afc9b747c4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:21:13.048851 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.048579    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-service-ca" (OuterVolumeSpecName: "service-ca") pod "eb950baa-e857-4d57-a325-00afc9b747c4" (UID: "eb950baa-e857-4d57-a325-00afc9b747c4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:21:13.048851 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.048681    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-service-ca\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:13.048851 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.048700    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-console-config\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:13.048851 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.048708    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "eb950baa-e857-4d57-a325-00afc9b747c4" (UID: "eb950baa-e857-4d57-a325-00afc9b747c4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:21:13.049085 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.048921    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "eb950baa-e857-4d57-a325-00afc9b747c4" (UID: "eb950baa-e857-4d57-a325-00afc9b747c4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:21:13.050799 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.050766    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb950baa-e857-4d57-a325-00afc9b747c4-kube-api-access-nwxmq" (OuterVolumeSpecName: "kube-api-access-nwxmq") pod "eb950baa-e857-4d57-a325-00afc9b747c4" (UID: "eb950baa-e857-4d57-a325-00afc9b747c4"). InnerVolumeSpecName "kube-api-access-nwxmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:21:13.050799 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.050766    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb950baa-e857-4d57-a325-00afc9b747c4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "eb950baa-e857-4d57-a325-00afc9b747c4" (UID: "eb950baa-e857-4d57-a325-00afc9b747c4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:21:13.050928 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.050853    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb950baa-e857-4d57-a325-00afc9b747c4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "eb950baa-e857-4d57-a325-00afc9b747c4" (UID: "eb950baa-e857-4d57-a325-00afc9b747c4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:21:13.051336 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.051299    2559 scope.go:117] "RemoveContainer" containerID="17a255fcc36bd4446c703a807c362b9c8bb39f5e7cdb4449aebaab33dd20f7a0"
Jul 10 00:21:13.051582 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:21:13.051561    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17a255fcc36bd4446c703a807c362b9c8bb39f5e7cdb4449aebaab33dd20f7a0\": container with ID starting with 17a255fcc36bd4446c703a807c362b9c8bb39f5e7cdb4449aebaab33dd20f7a0 not found: ID does not exist" containerID="17a255fcc36bd4446c703a807c362b9c8bb39f5e7cdb4449aebaab33dd20f7a0"
Jul 10 00:21:13.051637 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.051592    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17a255fcc36bd4446c703a807c362b9c8bb39f5e7cdb4449aebaab33dd20f7a0"} err="failed to get container status \"17a255fcc36bd4446c703a807c362b9c8bb39f5e7cdb4449aebaab33dd20f7a0\": rpc error: code = NotFound desc = could not find container \"17a255fcc36bd4446c703a807c362b9c8bb39f5e7cdb4449aebaab33dd20f7a0\": container with ID starting with 17a255fcc36bd4446c703a807c362b9c8bb39f5e7cdb4449aebaab33dd20f7a0 not found: ID does not exist"
Jul 10 00:21:13.149871 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.149790    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb950baa-e857-4d57-a325-00afc9b747c4-console-oauth-config\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:13.149871 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.149816    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwxmq\" (UniqueName: \"kubernetes.io/projected/eb950baa-e857-4d57-a325-00afc9b747c4-kube-api-access-nwxmq\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:13.149871 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.149826    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-trusted-ca-bundle\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:13.149871 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.149835    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb950baa-e857-4d57-a325-00afc9b747c4-console-serving-cert\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:13.149871 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.149845    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb950baa-e857-4d57-a325-00afc9b747c4-oauth-serving-cert\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:13.325480 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.325453    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gphtp_7ba31ebb-c51d-4692-80de-d8a7384b7e2c/node-ca/0.log"
Jul 10 00:21:13.364186 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.364157    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b78844f9-ft8jm"]
Jul 10 00:21:13.372220 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:13.369120    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b78844f9-ft8jm"]
Jul 10 00:21:14.126282 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:14.126250    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vtm7t_1d2b0c9d-41f9-45be-a60c-52fa7cb19f15/serve-healthcheck-canary/0.log"
Jul 10 00:21:14.835081 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:14.835046    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb950baa-e857-4d57-a325-00afc9b747c4" path="/var/lib/kubelet/pods/eb950baa-e857-4d57-a325-00afc9b747c4/volumes"
Jul 10 00:21:15.431324 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:15.431277    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:15.527544 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:15.527504    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:22.666948 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:22.666913    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:22.768253 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:22.768213    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:22.828717 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:22.828687    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:24.608395 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:24.608358    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:24.628405 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:24.628380    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:25.119389 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:25.119361    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:28.611890 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:28.611856    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:28.660788 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:28.660753    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:29.413851 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:29.413818    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:29.465108 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:29.465074    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:30.427650 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:30.427611    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:30.528055 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:30.528024    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:34.105668 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:34.105633    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:34.158913 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:34.158881    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:40.039664 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:40.039632    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:40.092276 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:40.092243    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:40.585835 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:40.585803    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:40.648682 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:40.648651    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:40.715084 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:40.715051    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:41.318928 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:41.318897    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:41.376195 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:41.376164    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:42.566157 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:42.566065    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 00:21:42.566634 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:42.566467    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="prometheus" containerID="cri-o://f446c37f66beddbebd0524637f7dd7c2e9b6fade4f0766c5028b01ad8eda689c" gracePeriod=600
Jul 10 00:21:42.566634 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:42.566495    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://2f964d30ac92fa090acdb7922d5bed6f10ad8ac2534a8b01050a8cf39013b516" gracePeriod=600
Jul 10 00:21:42.566634 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:42.566495    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="kube-rbac-proxy" containerID="cri-o://ba6618e687500e09288705c60972d790c9a64447cd54801264cf62e158e8c2f9" gracePeriod=600
Jul 10 00:21:42.566634 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:42.566540    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="kube-rbac-proxy-web" containerID="cri-o://1cb59a65402f1465160fe8ad92706145c57a43f3bfdf6092880f0fc17a5c041d" gracePeriod=600
Jul 10 00:21:42.566634 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:42.566517    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="config-reloader" containerID="cri-o://886c6cb2fe6f36dddd14f93d51278d52d903eb0c79c42a375eed110a993bee6f" gracePeriod=600
Jul 10 00:21:42.566634 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:42.566540    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="thanos-sidecar" containerID="cri-o://79086510fea92853e03fff9a89e8a2ea1a4b315c2131c4307c4e0f5df63364c1" gracePeriod=600
Jul 10 00:21:43.168063 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.168029    2559 generic.go:356] "Generic (PLEG): container finished" podID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerID="2f964d30ac92fa090acdb7922d5bed6f10ad8ac2534a8b01050a8cf39013b516" exitCode=0
Jul 10 00:21:43.168063 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.168056    2559 generic.go:356] "Generic (PLEG): container finished" podID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerID="ba6618e687500e09288705c60972d790c9a64447cd54801264cf62e158e8c2f9" exitCode=0
Jul 10 00:21:43.168063 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.168063    2559 generic.go:356] "Generic (PLEG): container finished" podID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerID="79086510fea92853e03fff9a89e8a2ea1a4b315c2131c4307c4e0f5df63364c1" exitCode=0
Jul 10 00:21:43.168063 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.168069    2559 generic.go:356] "Generic (PLEG): container finished" podID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerID="886c6cb2fe6f36dddd14f93d51278d52d903eb0c79c42a375eed110a993bee6f" exitCode=0
Jul 10 00:21:43.168063 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.168074    2559 generic.go:356] "Generic (PLEG): container finished" podID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerID="f446c37f66beddbebd0524637f7dd7c2e9b6fade4f0766c5028b01ad8eda689c" exitCode=0
Jul 10 00:21:43.168462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.168103    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3937d9c7-89ab-4972-b3af-f40c98f3274c","Type":"ContainerDied","Data":"2f964d30ac92fa090acdb7922d5bed6f10ad8ac2534a8b01050a8cf39013b516"}
Jul 10 00:21:43.168462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.168219    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3937d9c7-89ab-4972-b3af-f40c98f3274c","Type":"ContainerDied","Data":"ba6618e687500e09288705c60972d790c9a64447cd54801264cf62e158e8c2f9"}
Jul 10 00:21:43.168462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.168261    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3937d9c7-89ab-4972-b3af-f40c98f3274c","Type":"ContainerDied","Data":"79086510fea92853e03fff9a89e8a2ea1a4b315c2131c4307c4e0f5df63364c1"}
Jul 10 00:21:43.168462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.168302    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3937d9c7-89ab-4972-b3af-f40c98f3274c","Type":"ContainerDied","Data":"886c6cb2fe6f36dddd14f93d51278d52d903eb0c79c42a375eed110a993bee6f"}
Jul 10 00:21:43.168462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.168365    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3937d9c7-89ab-4972-b3af-f40c98f3274c","Type":"ContainerDied","Data":"f446c37f66beddbebd0524637f7dd7c2e9b6fade4f0766c5028b01ad8eda689c"}
Jul 10 00:21:43.811437 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.811413    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:43.920542 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.920448    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-kubelet-serving-ca-bundle\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.920542 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.920491    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-grpc-tls\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.920542 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.920528    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.920810 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.920557    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-db\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.920810 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.920590    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.920810 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.920656    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-2\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.920810 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.920687    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-serving-certs-ca-bundle\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.920810 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.920724    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-1\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.920810 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.920763    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-0\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.920810 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.920787    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-tls\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.920810 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.920814    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-config\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.921158 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.920842    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3937d9c7-89ab-4972-b3af-f40c98f3274c-tls-assets\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.921158 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.920874    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-metrics-client-ca\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.921158 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.920914    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-metrics-client-certs\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.921158 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.920954    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-trusted-ca-bundle\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.921158 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.920987    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3937d9c7-89ab-4972-b3af-f40c98f3274c-config-out\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.921158 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.921011    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-thanos-prometheus-http-client-file\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.921158 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.921057    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4xfn\" (UniqueName: \"kubernetes.io/projected/3937d9c7-89ab-4972-b3af-f40c98f3274c-kube-api-access-z4xfn\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.921158 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.921087    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-kube-rbac-proxy\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.921158 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.921115    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-web-config\") pod \"3937d9c7-89ab-4972-b3af-f40c98f3274c\" (UID: \"3937d9c7-89ab-4972-b3af-f40c98f3274c\") "
Jul 10 00:21:43.921158 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.921134    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:21:43.921666 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.921231    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:21:43.921666 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.921494    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:21:43.921666 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.921554    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:43.921666 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.921577    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:43.921877 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.921802    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:21:43.921877 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.921843    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:21:43.922672 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.922638    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:21:43.922918 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.922890    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:21:43.923507 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.923481    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:21:43.923605 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.923518    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:21:43.923676 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.923623    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 00:21:43.924018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.923873    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:21:43.924229 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.924191    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:21:43.925215 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.925188    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:21:43.925657 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.925623    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-config" (OuterVolumeSpecName: "config") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:21:43.925798 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.925777    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:21:43.925971 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.925951    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3937d9c7-89ab-4972-b3af-f40c98f3274c-config-out" (OuterVolumeSpecName: "config-out") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 00:21:43.925971 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.925959    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3937d9c7-89ab-4972-b3af-f40c98f3274c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:21:43.926082 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.926002    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3937d9c7-89ab-4972-b3af-f40c98f3274c-kube-api-access-z4xfn" (OuterVolumeSpecName: "kube-api-access-z4xfn") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "kube-api-access-z4xfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:21:43.926082 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.926031    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:21:43.936795 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:43.936775    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-web-config" (OuterVolumeSpecName: "web-config") pod "3937d9c7-89ab-4972-b3af-f40c98f3274c" (UID: "3937d9c7-89ab-4972-b3af-f40c98f3274c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:21:44.022397 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022363    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022397 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022392    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022397 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022402    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022412    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022422    2559 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-config\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022430    2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3937d9c7-89ab-4972-b3af-f40c98f3274c-tls-assets\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022439    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-metrics-client-ca\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022446    2559 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-metrics-client-certs\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022455    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022463    2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3937d9c7-89ab-4972-b3af-f40c98f3274c-config-out\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022471    2559 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022480    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4xfn\" (UniqueName: \"kubernetes.io/projected/3937d9c7-89ab-4972-b3af-f40c98f3274c-kube-api-access-z4xfn\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022488    2559 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-kube-rbac-proxy\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022497    2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-web-config\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022505    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3937d9c7-89ab-4972-b3af-f40c98f3274c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022513    2559 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-grpc-tls\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022521    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3937d9c7-89ab-4972-b3af-f40c98f3274c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.022629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.022530    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3937d9c7-89ab-4972-b3af-f40c98f3274c-prometheus-k8s-db\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:21:44.175393 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.175290    2559 generic.go:356] "Generic (PLEG): container finished" podID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerID="1cb59a65402f1465160fe8ad92706145c57a43f3bfdf6092880f0fc17a5c041d" exitCode=0
Jul 10 00:21:44.175393 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.175340    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3937d9c7-89ab-4972-b3af-f40c98f3274c","Type":"ContainerDied","Data":"1cb59a65402f1465160fe8ad92706145c57a43f3bfdf6092880f0fc17a5c041d"}
Jul 10 00:21:44.175566 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.175400    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.175566 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.175415    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3937d9c7-89ab-4972-b3af-f40c98f3274c","Type":"ContainerDied","Data":"54d6ce8a51f54f8e22dd0e71d80607820b6b4caa74576b7e65035d654f5ba09f"}
Jul 10 00:21:44.175566 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.175448    2559 scope.go:117] "RemoveContainer" containerID="2f964d30ac92fa090acdb7922d5bed6f10ad8ac2534a8b01050a8cf39013b516"
Jul 10 00:21:44.183246 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.183213    2559 scope.go:117] "RemoveContainer" containerID="ba6618e687500e09288705c60972d790c9a64447cd54801264cf62e158e8c2f9"
Jul 10 00:21:44.189988 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.189971    2559 scope.go:117] "RemoveContainer" containerID="1cb59a65402f1465160fe8ad92706145c57a43f3bfdf6092880f0fc17a5c041d"
Jul 10 00:21:44.196300 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.196279    2559 scope.go:117] "RemoveContainer" containerID="79086510fea92853e03fff9a89e8a2ea1a4b315c2131c4307c4e0f5df63364c1"
Jul 10 00:21:44.199038 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.199016    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 00:21:44.203637 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.203624    2559 scope.go:117] "RemoveContainer" containerID="886c6cb2fe6f36dddd14f93d51278d52d903eb0c79c42a375eed110a993bee6f"
Jul 10 00:21:44.204106 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.204087    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 00:21:44.210207 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.210189    2559 scope.go:117] "RemoveContainer" containerID="f446c37f66beddbebd0524637f7dd7c2e9b6fade4f0766c5028b01ad8eda689c"
Jul 10 00:21:44.216730 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.216714    2559 scope.go:117] "RemoveContainer" containerID="5f5d57eec3459e97cf1c8b700f47211aa8c5b6080339ec12e87aee7e328cbbc4"
Jul 10 00:21:44.223117 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.223099    2559 scope.go:117] "RemoveContainer" containerID="2f964d30ac92fa090acdb7922d5bed6f10ad8ac2534a8b01050a8cf39013b516"
Jul 10 00:21:44.223400 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:21:44.223382    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f964d30ac92fa090acdb7922d5bed6f10ad8ac2534a8b01050a8cf39013b516\": container with ID starting with 2f964d30ac92fa090acdb7922d5bed6f10ad8ac2534a8b01050a8cf39013b516 not found: ID does not exist" containerID="2f964d30ac92fa090acdb7922d5bed6f10ad8ac2534a8b01050a8cf39013b516"
Jul 10 00:21:44.223481 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.223409    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f964d30ac92fa090acdb7922d5bed6f10ad8ac2534a8b01050a8cf39013b516"} err="failed to get container status \"2f964d30ac92fa090acdb7922d5bed6f10ad8ac2534a8b01050a8cf39013b516\": rpc error: code = NotFound desc = could not find container \"2f964d30ac92fa090acdb7922d5bed6f10ad8ac2534a8b01050a8cf39013b516\": container with ID starting with 2f964d30ac92fa090acdb7922d5bed6f10ad8ac2534a8b01050a8cf39013b516 not found: ID does not exist"
Jul 10 00:21:44.223481 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.223427    2559 scope.go:117] "RemoveContainer" containerID="ba6618e687500e09288705c60972d790c9a64447cd54801264cf62e158e8c2f9"
Jul 10 00:21:44.223677 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:21:44.223660    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba6618e687500e09288705c60972d790c9a64447cd54801264cf62e158e8c2f9\": container with ID starting with ba6618e687500e09288705c60972d790c9a64447cd54801264cf62e158e8c2f9 not found: ID does not exist" containerID="ba6618e687500e09288705c60972d790c9a64447cd54801264cf62e158e8c2f9"
Jul 10 00:21:44.223718 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.223683    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba6618e687500e09288705c60972d790c9a64447cd54801264cf62e158e8c2f9"} err="failed to get container status \"ba6618e687500e09288705c60972d790c9a64447cd54801264cf62e158e8c2f9\": rpc error: code = NotFound desc = could not find container \"ba6618e687500e09288705c60972d790c9a64447cd54801264cf62e158e8c2f9\": container with ID starting with ba6618e687500e09288705c60972d790c9a64447cd54801264cf62e158e8c2f9 not found: ID does not exist"
Jul 10 00:21:44.223718 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.223699    2559 scope.go:117] "RemoveContainer" containerID="1cb59a65402f1465160fe8ad92706145c57a43f3bfdf6092880f0fc17a5c041d"
Jul 10 00:21:44.223965 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:21:44.223943    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cb59a65402f1465160fe8ad92706145c57a43f3bfdf6092880f0fc17a5c041d\": container with ID starting with 1cb59a65402f1465160fe8ad92706145c57a43f3bfdf6092880f0fc17a5c041d not found: ID does not exist" containerID="1cb59a65402f1465160fe8ad92706145c57a43f3bfdf6092880f0fc17a5c041d"
Jul 10 00:21:44.224027 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.223974    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb59a65402f1465160fe8ad92706145c57a43f3bfdf6092880f0fc17a5c041d"} err="failed to get container status \"1cb59a65402f1465160fe8ad92706145c57a43f3bfdf6092880f0fc17a5c041d\": rpc error: code = NotFound desc = could not find container \"1cb59a65402f1465160fe8ad92706145c57a43f3bfdf6092880f0fc17a5c041d\": container with ID starting with 1cb59a65402f1465160fe8ad92706145c57a43f3bfdf6092880f0fc17a5c041d not found: ID does not exist"
Jul 10 00:21:44.224027 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.223995    2559 scope.go:117] "RemoveContainer" containerID="79086510fea92853e03fff9a89e8a2ea1a4b315c2131c4307c4e0f5df63364c1"
Jul 10 00:21:44.224246 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:21:44.224227    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79086510fea92853e03fff9a89e8a2ea1a4b315c2131c4307c4e0f5df63364c1\": container with ID starting with 79086510fea92853e03fff9a89e8a2ea1a4b315c2131c4307c4e0f5df63364c1 not found: ID does not exist" containerID="79086510fea92853e03fff9a89e8a2ea1a4b315c2131c4307c4e0f5df63364c1"
Jul 10 00:21:44.224328 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.224259    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79086510fea92853e03fff9a89e8a2ea1a4b315c2131c4307c4e0f5df63364c1"} err="failed to get container status \"79086510fea92853e03fff9a89e8a2ea1a4b315c2131c4307c4e0f5df63364c1\": rpc error: code = NotFound desc = could not find container \"79086510fea92853e03fff9a89e8a2ea1a4b315c2131c4307c4e0f5df63364c1\": container with ID starting with 79086510fea92853e03fff9a89e8a2ea1a4b315c2131c4307c4e0f5df63364c1 not found: ID does not exist"
Jul 10 00:21:44.224328 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.224280    2559 scope.go:117] "RemoveContainer" containerID="886c6cb2fe6f36dddd14f93d51278d52d903eb0c79c42a375eed110a993bee6f"
Jul 10 00:21:44.224641 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:21:44.224615    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"886c6cb2fe6f36dddd14f93d51278d52d903eb0c79c42a375eed110a993bee6f\": container with ID starting with 886c6cb2fe6f36dddd14f93d51278d52d903eb0c79c42a375eed110a993bee6f not found: ID does not exist" containerID="886c6cb2fe6f36dddd14f93d51278d52d903eb0c79c42a375eed110a993bee6f"
Jul 10 00:21:44.224741 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.224646    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"886c6cb2fe6f36dddd14f93d51278d52d903eb0c79c42a375eed110a993bee6f"} err="failed to get container status \"886c6cb2fe6f36dddd14f93d51278d52d903eb0c79c42a375eed110a993bee6f\": rpc error: code = NotFound desc = could not find container \"886c6cb2fe6f36dddd14f93d51278d52d903eb0c79c42a375eed110a993bee6f\": container with ID starting with 886c6cb2fe6f36dddd14f93d51278d52d903eb0c79c42a375eed110a993bee6f not found: ID does not exist"
Jul 10 00:21:44.224741 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.224672    2559 scope.go:117] "RemoveContainer" containerID="f446c37f66beddbebd0524637f7dd7c2e9b6fade4f0766c5028b01ad8eda689c"
Jul 10 00:21:44.225058 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:21:44.225037    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f446c37f66beddbebd0524637f7dd7c2e9b6fade4f0766c5028b01ad8eda689c\": container with ID starting with f446c37f66beddbebd0524637f7dd7c2e9b6fade4f0766c5028b01ad8eda689c not found: ID does not exist" containerID="f446c37f66beddbebd0524637f7dd7c2e9b6fade4f0766c5028b01ad8eda689c"
Jul 10 00:21:44.225147 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.225092    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f446c37f66beddbebd0524637f7dd7c2e9b6fade4f0766c5028b01ad8eda689c"} err="failed to get container status \"f446c37f66beddbebd0524637f7dd7c2e9b6fade4f0766c5028b01ad8eda689c\": rpc error: code = NotFound desc = could not find container \"f446c37f66beddbebd0524637f7dd7c2e9b6fade4f0766c5028b01ad8eda689c\": container with ID starting with f446c37f66beddbebd0524637f7dd7c2e9b6fade4f0766c5028b01ad8eda689c not found: ID does not exist"
Jul 10 00:21:44.225147 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.225110    2559 scope.go:117] "RemoveContainer" containerID="5f5d57eec3459e97cf1c8b700f47211aa8c5b6080339ec12e87aee7e328cbbc4"
Jul 10 00:21:44.225394 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:21:44.225375    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f5d57eec3459e97cf1c8b700f47211aa8c5b6080339ec12e87aee7e328cbbc4\": container with ID starting with 5f5d57eec3459e97cf1c8b700f47211aa8c5b6080339ec12e87aee7e328cbbc4 not found: ID does not exist" containerID="5f5d57eec3459e97cf1c8b700f47211aa8c5b6080339ec12e87aee7e328cbbc4"
Jul 10 00:21:44.225455 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.225397    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f5d57eec3459e97cf1c8b700f47211aa8c5b6080339ec12e87aee7e328cbbc4"} err="failed to get container status \"5f5d57eec3459e97cf1c8b700f47211aa8c5b6080339ec12e87aee7e328cbbc4\": rpc error: code = NotFound desc = could not find container \"5f5d57eec3459e97cf1c8b700f47211aa8c5b6080339ec12e87aee7e328cbbc4\": container with ID starting with 5f5d57eec3459e97cf1c8b700f47211aa8c5b6080339ec12e87aee7e328cbbc4 not found: ID does not exist"
Jul 10 00:21:44.226094 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226077    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 00:21:44.226375 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226363    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="thanos-sidecar"
Jul 10 00:21:44.226418 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226378    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="thanos-sidecar"
Jul 10 00:21:44.226418 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226391    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="prometheus"
Jul 10 00:21:44.226418 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226396    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="prometheus"
Jul 10 00:21:44.226418 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226405    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="kube-rbac-proxy-web"
Jul 10 00:21:44.226418 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226414    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="kube-rbac-proxy-web"
Jul 10 00:21:44.226565 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226424    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="init-config-reloader"
Jul 10 00:21:44.226565 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226429    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="init-config-reloader"
Jul 10 00:21:44.226565 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226435    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb950baa-e857-4d57-a325-00afc9b747c4" containerName="console"
Jul 10 00:21:44.226565 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226440    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb950baa-e857-4d57-a325-00afc9b747c4" containerName="console"
Jul 10 00:21:44.226565 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226451    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="config-reloader"
Jul 10 00:21:44.226565 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226456    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="config-reloader"
Jul 10 00:21:44.226565 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226464    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="kube-rbac-proxy"
Jul 10 00:21:44.226565 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226469    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="kube-rbac-proxy"
Jul 10 00:21:44.226565 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226477    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4703c32b-78f0-4cbc-9461-aa323d3fcf67" containerName="registry"
Jul 10 00:21:44.226565 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226484    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4703c32b-78f0-4cbc-9461-aa323d3fcf67" containerName="registry"
Jul 10 00:21:44.226565 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226496    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="kube-rbac-proxy-thanos"
Jul 10 00:21:44.226565 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226505    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="kube-rbac-proxy-thanos"
Jul 10 00:21:44.226565 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226562    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="config-reloader"
Jul 10 00:21:44.226968 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226572    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="thanos-sidecar"
Jul 10 00:21:44.226968 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226580    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="kube-rbac-proxy-thanos"
Jul 10 00:21:44.226968 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226587    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="prometheus"
Jul 10 00:21:44.226968 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226593    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4703c32b-78f0-4cbc-9461-aa323d3fcf67" containerName="registry"
Jul 10 00:21:44.226968 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226599    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb950baa-e857-4d57-a325-00afc9b747c4" containerName="console"
Jul 10 00:21:44.226968 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226603    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="kube-rbac-proxy-web"
Jul 10 00:21:44.226968 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.226608    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" containerName="kube-rbac-proxy"
Jul 10 00:21:44.231577 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.231552    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.234291 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.234268    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 10 00:21:44.234412 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.234275    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 10 00:21:44.234728 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.234709    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 10 00:21:44.235145 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.235082    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-41uo784l26j6d\""
Jul 10 00:21:44.235221 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.235141    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zc5b7\""
Jul 10 00:21:44.235221 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.235190    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 10 00:21:44.235526 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.235512    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 10 00:21:44.235580 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.235530    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 10 00:21:44.235580 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.235558    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 10 00:21:44.235999 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.235980    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 10 00:21:44.236083 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.236031    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 10 00:21:44.236083 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.236044    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 10 00:21:44.236185 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.236116    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 10 00:21:44.236185 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.236146    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 10 00:21:44.236679 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.236662    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 10 00:21:44.239008 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.238989    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 10 00:21:44.240249 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.240228    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 10 00:21:44.244111 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.244092    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 00:21:44.324819 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.324790    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325013 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.324826    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cf3e8174-678c-472a-b5b6-5ca9235e10ef-config-out\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325013 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.324852    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f58l9\" (UniqueName: \"kubernetes.io/projected/cf3e8174-678c-472a-b5b6-5ca9235e10ef-kube-api-access-f58l9\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325013 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.324909    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325013 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.324951    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cf3e8174-678c-472a-b5b6-5ca9235e10ef-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325013 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.325008    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325388 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.325033    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325388 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.325059    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325388 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.325084    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325388 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.325114    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325388 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.325144    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325388 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.325168    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325388 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.325215    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325388 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.325255    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325388 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.325287    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-config\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325388 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.325337    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325388 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.325372    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325759 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.325409    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325759 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.325434    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-web-config\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.325759 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.325458    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cf3e8174-678c-472a-b5b6-5ca9235e10ef-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426108 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426040    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426108 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426074    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cf3e8174-678c-472a-b5b6-5ca9235e10ef-config-out\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426108 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426097    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f58l9\" (UniqueName: \"kubernetes.io/projected/cf3e8174-678c-472a-b5b6-5ca9235e10ef-kube-api-access-f58l9\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426305 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426202    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426305 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426234    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cf3e8174-678c-472a-b5b6-5ca9235e10ef-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426305 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426259    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426305 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426279    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426305 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426299    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426567 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426342    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426567 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426372    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426567 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426403    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426567 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426428    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426567 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426461    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426567 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426491    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426567 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426526    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-config\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426567 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426551    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426963 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426586    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426963 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426630    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426963 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426654    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-web-config\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426963 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426678    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cf3e8174-678c-472a-b5b6-5ca9235e10ef-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426963 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426703    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cf3e8174-678c-472a-b5b6-5ca9235e10ef-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.426963 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426896    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.427339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.426991    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.429195 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.429160    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.429300 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.429234    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.429425 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.429391    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.429425 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.429399    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.429609 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.429539    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.429945 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.429920    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.430036 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.429939    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.430036 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.429959    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.430151 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.430033    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.430524 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.430461    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.430793 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.430765    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cf3e8174-678c-472a-b5b6-5ca9235e10ef-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.431240 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.431217    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cf3e8174-678c-472a-b5b6-5ca9235e10ef-config-out\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.431567 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.431547    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-web-config\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.431723 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.431704    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.431998 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.431982    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cf3e8174-678c-472a-b5b6-5ca9235e10ef-config\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.432035 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.432003    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cf3e8174-678c-472a-b5b6-5ca9235e10ef-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.434202 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.434185    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f58l9\" (UniqueName: \"kubernetes.io/projected/cf3e8174-678c-472a-b5b6-5ca9235e10ef-kube-api-access-f58l9\") pod \"prometheus-k8s-0\" (UID: \"cf3e8174-678c-472a-b5b6-5ca9235e10ef\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.542512 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.542475    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:44.677386 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.677282    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 00:21:44.680626 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:21:44.680584    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf3e8174_678c_472a_b5b6_5ca9235e10ef.slice/crio-43037f4a46c0a7289d0331c6b843fc00534fa730bdc5e28bfbd06053e0628dc0 WatchSource:0}: Error finding container 43037f4a46c0a7289d0331c6b843fc00534fa730bdc5e28bfbd06053e0628dc0: Status 404 returned error can't find the container with id 43037f4a46c0a7289d0331c6b843fc00534fa730bdc5e28bfbd06053e0628dc0
Jul 10 00:21:44.837149 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:44.837112    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3937d9c7-89ab-4972-b3af-f40c98f3274c" path="/var/lib/kubelet/pods/3937d9c7-89ab-4972-b3af-f40c98f3274c/volumes"
Jul 10 00:21:45.180937 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:45.180902    2559 generic.go:356] "Generic (PLEG): container finished" podID="cf3e8174-678c-472a-b5b6-5ca9235e10ef" containerID="adfa90c1df39324345bbec6d28bc6cbbeb754c1208f98f6f74ec503faf17e537" exitCode=0
Jul 10 00:21:45.181119 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:45.180985    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cf3e8174-678c-472a-b5b6-5ca9235e10ef","Type":"ContainerDied","Data":"adfa90c1df39324345bbec6d28bc6cbbeb754c1208f98f6f74ec503faf17e537"}
Jul 10 00:21:45.181119 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:45.181051    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cf3e8174-678c-472a-b5b6-5ca9235e10ef","Type":"ContainerStarted","Data":"43037f4a46c0a7289d0331c6b843fc00534fa730bdc5e28bfbd06053e0628dc0"}
Jul 10 00:21:45.435098 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:45.435057    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:45.535450 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:45.535415    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:45.636033 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:45.636003    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:46.189643 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:46.189609    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cf3e8174-678c-472a-b5b6-5ca9235e10ef","Type":"ContainerStarted","Data":"487f9cc0edc9e147e33b1aee45b278a94338286d168355cf858d3be1c5a8d155"}
Jul 10 00:21:46.190017 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:46.189712    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cf3e8174-678c-472a-b5b6-5ca9235e10ef","Type":"ContainerStarted","Data":"9122154d33fad4fd128ba4547525dca8308f90beeae37110a1e93271bf1bd51a"}
Jul 10 00:21:46.190017 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:46.189751    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cf3e8174-678c-472a-b5b6-5ca9235e10ef","Type":"ContainerStarted","Data":"93cf8205a25b4ddb635c1bf660d6ce871c88a3c215aa3ba931d8cd2567b0db23"}
Jul 10 00:21:46.190017 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:46.189777    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cf3e8174-678c-472a-b5b6-5ca9235e10ef","Type":"ContainerStarted","Data":"53209dfe8f360c3b4ed2c8b5ff0a675c4a908d5aa203a7079b463ebe669b181b"}
Jul 10 00:21:46.190017 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:46.189808    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cf3e8174-678c-472a-b5b6-5ca9235e10ef","Type":"ContainerStarted","Data":"740ce4927b11720f90bc986a0d092552c2d6fbac1ad9731a302740c9c9239419"}
Jul 10 00:21:46.190017 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:46.189832    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cf3e8174-678c-472a-b5b6-5ca9235e10ef","Type":"ContainerStarted","Data":"eb22a128f843fd56e6c0bc6c5ec256f2f90cee55fbf395ff3edab28deba86ad3"}
Jul 10 00:21:46.219702 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:46.218631    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.199122921 podStartE2EDuration="2.218609948s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.487027ms" isStatefulPod=false podCreationTimestamp="2026-07-10 00:21:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:21:46.21610404 +0000 UTC m=+355.918888909" watchObservedRunningTime="2026-07-10 00:21:46.218609948 +0000 UTC m=+355.921394799"
Jul 10 00:21:49.542984 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:49.542953    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:21:50.109149 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:50.109112    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:50.161265 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:50.161234    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:51.546055 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:51.546009    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:51.599181 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:51.599147    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:21:51.647172 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:21:51.647135    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:22:05.897032 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:22:05.896996    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:22:08.425503 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:22:08.425469    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:22:08.485487 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:22:08.485457    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:22:36.382576 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:22:36.382544    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:22:36.479666 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:22:36.479635    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:22:42.778004 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:22:42.777957    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:22:42.827482 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:22:42.827450    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:22:44.542742 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:22:44.542700    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:22:44.558131 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:22:44.558104    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:22:45.422629 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:22:45.422598    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 00:22:46.170865 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:22:46.170836    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:22:46.268837 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:22:46.268780    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:02.813646 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:02.813612    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:04.187095 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:04.187063    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:06.421514 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:06.421477    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:06.521612 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:06.521571    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:07.726109 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:07.726074    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:07.911281 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:07.911249    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:08.490863 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:08.490832    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:08.589084 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:08.589042    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:10.488599 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:10.488565    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:10.868291 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:10.868197    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:13.220343 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:13.220292    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:13.274302 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:13.274273    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:16.754673 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:16.754641    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:16.808799 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:16.808771    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:19.075639 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:19.075603    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:24.818887 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:24.818856    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:24.919604 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:24.919573    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:31.177605 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:31.177569    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:31.278413 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:31.278382    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:33.063788 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:33.063753    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:46.028960 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:46.028926    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:46.123325 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:46.123293    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:46.223723 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:46.223688    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:46.833801 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:46.833769    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:47.037492 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.037459    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:47.134688 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.134590    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:47.172207 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.172180    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"]
Jul 10 00:23:47.174665 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.174647    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.177282 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.177257    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 10 00:23:47.177467 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.177324    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 10 00:23:47.177467 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.177347    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 10 00:23:47.177467 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.177353    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hdnzg\""
Jul 10 00:23:47.177467 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.177411    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 10 00:23:47.194000 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.193963    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"]
Jul 10 00:23:47.293516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.293474    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:47.367868 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.367835    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c55e31dc-0eac-449a-9d29-5805df85d1ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-559fc4999-zghf9\" (UID: \"c55e31dc-0eac-449a-9d29-5805df85d1ce\") " pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.367868 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.367868    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c55e31dc-0eac-449a-9d29-5805df85d1ce-manifests\") pod \"opendatahub-operator-controller-manager-559fc4999-zghf9\" (UID: \"c55e31dc-0eac-449a-9d29-5805df85d1ce\") " pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.368077 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.367890    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c55e31dc-0eac-449a-9d29-5805df85d1ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-559fc4999-zghf9\" (UID: \"c55e31dc-0eac-449a-9d29-5805df85d1ce\") " pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.368077 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.367979    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c55e31dc-0eac-449a-9d29-5805df85d1ce-tmp\") pod \"opendatahub-operator-controller-manager-559fc4999-zghf9\" (UID: \"c55e31dc-0eac-449a-9d29-5805df85d1ce\") " pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.368077 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.368032    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcsqg\" (UniqueName: \"kubernetes.io/projected/c55e31dc-0eac-449a-9d29-5805df85d1ce-kube-api-access-zcsqg\") pod \"opendatahub-operator-controller-manager-559fc4999-zghf9\" (UID: \"c55e31dc-0eac-449a-9d29-5805df85d1ce\") " pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.406056 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.405957    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:47.468749 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.468714    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c55e31dc-0eac-449a-9d29-5805df85d1ce-tmp\") pod \"opendatahub-operator-controller-manager-559fc4999-zghf9\" (UID: \"c55e31dc-0eac-449a-9d29-5805df85d1ce\") " pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.468903 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.468768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcsqg\" (UniqueName: \"kubernetes.io/projected/c55e31dc-0eac-449a-9d29-5805df85d1ce-kube-api-access-zcsqg\") pod \"opendatahub-operator-controller-manager-559fc4999-zghf9\" (UID: \"c55e31dc-0eac-449a-9d29-5805df85d1ce\") " pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.468903 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.468811    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c55e31dc-0eac-449a-9d29-5805df85d1ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-559fc4999-zghf9\" (UID: \"c55e31dc-0eac-449a-9d29-5805df85d1ce\") " pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.468903 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.468832    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c55e31dc-0eac-449a-9d29-5805df85d1ce-manifests\") pod \"opendatahub-operator-controller-manager-559fc4999-zghf9\" (UID: \"c55e31dc-0eac-449a-9d29-5805df85d1ce\") " pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.468903 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.468849    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c55e31dc-0eac-449a-9d29-5805df85d1ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-559fc4999-zghf9\" (UID: \"c55e31dc-0eac-449a-9d29-5805df85d1ce\") " pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.469220 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.469198    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c55e31dc-0eac-449a-9d29-5805df85d1ce-manifests\") pod \"opendatahub-operator-controller-manager-559fc4999-zghf9\" (UID: \"c55e31dc-0eac-449a-9d29-5805df85d1ce\") " pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.471134 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.471111    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c55e31dc-0eac-449a-9d29-5805df85d1ce-tmp\") pod \"opendatahub-operator-controller-manager-559fc4999-zghf9\" (UID: \"c55e31dc-0eac-449a-9d29-5805df85d1ce\") " pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.471228 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.471190    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c55e31dc-0eac-449a-9d29-5805df85d1ce-webhook-cert\") pod \"opendatahub-operator-controller-manager-559fc4999-zghf9\" (UID: \"c55e31dc-0eac-449a-9d29-5805df85d1ce\") " pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.471287 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.471273    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c55e31dc-0eac-449a-9d29-5805df85d1ce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-559fc4999-zghf9\" (UID: \"c55e31dc-0eac-449a-9d29-5805df85d1ce\") " pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.477776 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.477745    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcsqg\" (UniqueName: \"kubernetes.io/projected/c55e31dc-0eac-449a-9d29-5805df85d1ce-kube-api-access-zcsqg\") pod \"opendatahub-operator-controller-manager-559fc4999-zghf9\" (UID: \"c55e31dc-0eac-449a-9d29-5805df85d1ce\") " pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.485178 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.485156    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:47.489037 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.489008    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:47.589832 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.589766    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:47.626279 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.626251    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"]
Jul 10 00:23:47.630000 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:23:47.629975    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc55e31dc_0eac_449a_9d29_5805df85d1ce.slice/crio-cb5f7d5a622f65b89b43ed98676a0e30f100a3773b655c5d947b1391131b7909 WatchSource:0}: Error finding container cb5f7d5a622f65b89b43ed98676a0e30f100a3773b655c5d947b1391131b7909: Status 404 returned error can't find the container with id cb5f7d5a622f65b89b43ed98676a0e30f100a3773b655c5d947b1391131b7909
Jul 10 00:23:47.632010 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.631991    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 00:23:47.670928 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:47.670859    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:48.643862 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:48.643823    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9" event={"ID":"c55e31dc-0eac-449a-9d29-5805df85d1ce","Type":"ContainerStarted","Data":"cb5f7d5a622f65b89b43ed98676a0e30f100a3773b655c5d947b1391131b7909"}
Jul 10 00:23:51.341360 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:51.341302    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:51.441903 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:51.441872    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:51.660373 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:51.660272    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9" event={"ID":"c55e31dc-0eac-449a-9d29-5805df85d1ce","Type":"ContainerStarted","Data":"9dfa8ad8e76cd26b5ed7ba38885e2c4c974c2bcb7f7650a082f1554cd38a8d5d"}
Jul 10 00:23:52.665902 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:52.665868    2559 generic.go:356] "Generic (PLEG): container finished" podID="c55e31dc-0eac-449a-9d29-5805df85d1ce" containerID="9dfa8ad8e76cd26b5ed7ba38885e2c4c974c2bcb7f7650a082f1554cd38a8d5d" exitCode=0
Jul 10 00:23:52.666369 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:52.665955    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9" event={"ID":"c55e31dc-0eac-449a-9d29-5805df85d1ce","Type":"ContainerDied","Data":"9dfa8ad8e76cd26b5ed7ba38885e2c4c974c2bcb7f7650a082f1554cd38a8d5d"}
Jul 10 00:23:53.671521 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:53.671481    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9" event={"ID":"c55e31dc-0eac-449a-9d29-5805df85d1ce","Type":"ContainerStarted","Data":"2df001418a9d90e3cb54a10c772033f207e9000e5e14d6b722d0d97a5b3d255e"}
Jul 10 00:23:53.671918 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:53.671616    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:23:53.695099 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:53.695048    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9" podStartSLOduration=2.2507231389999998 podStartE2EDuration="6.69503493s" totalImagesPullingTime="3.437357725s" totalInitContainerRuntime="1.006954066s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:23:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:23:53.693718372 +0000 UTC m=+483.396503241" watchObservedRunningTime="2026-07-10 00:23:53.69503493 +0000 UTC m=+483.397819778"
Jul 10 00:23:58.381505 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:58.381474    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:23:58.500034 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:23:58.500003    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:04.678634 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:04.678602    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-559fc4999-zghf9"
Jul 10 00:24:06.370397 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:06.370364    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:06.488978 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:06.488945    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:06.666973 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:06.666866    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:06.854656 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:06.854624    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:07.102040 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:07.102004    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:07.194949 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:07.194910    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:07.786886 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:07.786856    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:07.848884 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:07.848848    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-snjhz"]
Jul 10 00:24:07.860082 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:07.860058    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-snjhz"]
Jul 10 00:24:07.860223 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:07.860151    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-snjhz"
Jul 10 00:24:07.862945 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:07.862913    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 10 00:24:07.863084 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:07.863046    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-d8kx6\""
Jul 10 00:24:07.885869 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:07.885841    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:07.940248 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:07.940217    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ed16b45-2f07-415b-be68-9d0da6cf6277-tls-certs\") pod \"model-serving-api-54d89bd5fc-snjhz\" (UID: \"3ed16b45-2f07-415b-be68-9d0da6cf6277\") " pod="opendatahub/model-serving-api-54d89bd5fc-snjhz"
Jul 10 00:24:07.940416 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:07.940262    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87jkf\" (UniqueName: \"kubernetes.io/projected/3ed16b45-2f07-415b-be68-9d0da6cf6277-kube-api-access-87jkf\") pod \"model-serving-api-54d89bd5fc-snjhz\" (UID: \"3ed16b45-2f07-415b-be68-9d0da6cf6277\") " pod="opendatahub/model-serving-api-54d89bd5fc-snjhz"
Jul 10 00:24:07.966848 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:07.966816    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:08.038578 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:08.038497    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:08.041621 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:08.041597    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ed16b45-2f07-415b-be68-9d0da6cf6277-tls-certs\") pod \"model-serving-api-54d89bd5fc-snjhz\" (UID: \"3ed16b45-2f07-415b-be68-9d0da6cf6277\") " pod="opendatahub/model-serving-api-54d89bd5fc-snjhz"
Jul 10 00:24:08.041716 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:08.041657    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87jkf\" (UniqueName: \"kubernetes.io/projected/3ed16b45-2f07-415b-be68-9d0da6cf6277-kube-api-access-87jkf\") pod \"model-serving-api-54d89bd5fc-snjhz\" (UID: \"3ed16b45-2f07-415b-be68-9d0da6cf6277\") " pod="opendatahub/model-serving-api-54d89bd5fc-snjhz"
Jul 10 00:24:08.044186 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:08.044164    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ed16b45-2f07-415b-be68-9d0da6cf6277-tls-certs\") pod \"model-serving-api-54d89bd5fc-snjhz\" (UID: \"3ed16b45-2f07-415b-be68-9d0da6cf6277\") " pod="opendatahub/model-serving-api-54d89bd5fc-snjhz"
Jul 10 00:24:08.062102 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:08.062077    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87jkf\" (UniqueName: \"kubernetes.io/projected/3ed16b45-2f07-415b-be68-9d0da6cf6277-kube-api-access-87jkf\") pod \"model-serving-api-54d89bd5fc-snjhz\" (UID: \"3ed16b45-2f07-415b-be68-9d0da6cf6277\") " pod="opendatahub/model-serving-api-54d89bd5fc-snjhz"
Jul 10 00:24:08.154697 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:08.154663    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:08.171599 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:08.171577    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-snjhz"
Jul 10 00:24:08.295990 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:08.295909    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-snjhz"]
Jul 10 00:24:08.299074 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:24:08.299045    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ed16b45_2f07_415b_be68_9d0da6cf6277.slice/crio-371f2d3b0b48941d5a1dff4e3804c9fc2fbf158a3909cb8648ba0c4bc51902e9 WatchSource:0}: Error finding container 371f2d3b0b48941d5a1dff4e3804c9fc2fbf158a3909cb8648ba0c4bc51902e9: Status 404 returned error can't find the container with id 371f2d3b0b48941d5a1dff4e3804c9fc2fbf158a3909cb8648ba0c4bc51902e9
Jul 10 00:24:08.734997 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:08.734961    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-snjhz" event={"ID":"3ed16b45-2f07-415b-be68-9d0da6cf6277","Type":"ContainerStarted","Data":"371f2d3b0b48941d5a1dff4e3804c9fc2fbf158a3909cb8648ba0c4bc51902e9"}
Jul 10 00:24:09.049372 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:09.049276    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:10.748380 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:10.748344    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-snjhz" event={"ID":"3ed16b45-2f07-415b-be68-9d0da6cf6277","Type":"ContainerStarted","Data":"596db5d0881956f2396d5b9ec3035e0d6ae1b45a45dc01b148f3f5ecdc977dba"}
Jul 10 00:24:10.748724 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:10.748460    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-snjhz"
Jul 10 00:24:10.764717 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:10.764670    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-snjhz" podStartSLOduration=1.720577414 podStartE2EDuration="3.764655576s" totalImagesPullingTime="2.044078162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:24:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:24:10.763647699 +0000 UTC m=+500.466432573" watchObservedRunningTime="2026-07-10 00:24:10.764655576 +0000 UTC m=+500.467440583"
Jul 10 00:24:13.945150 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:13.945115    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:14.069396 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.069365    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:14.427348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.427254    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"]
Jul 10 00:24:14.430520 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.430504    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.433056 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.433034    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 00:24:14.433208 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.433135    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 10 00:24:14.433208 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.433183    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 10 00:24:14.433377 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.433215    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 10 00:24:14.433377 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.433233    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 10 00:24:14.433377 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.433260    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-v775r\""
Jul 10 00:24:14.433493 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.433449    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 10 00:24:14.445976 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.445947    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"]
Jul 10 00:24:14.496132 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.496110    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.496252 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.496147    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.496252 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.496177    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.496252 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.496197    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjx8q\" (UniqueName: \"kubernetes.io/projected/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-kube-api-access-fjx8q\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.496252 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.496241    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.496408 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.496270    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.496408 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.496294    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.597103 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.597069    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.597278 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.597127    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.597278 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.597146    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjx8q\" (UniqueName: \"kubernetes.io/projected/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-kube-api-access-fjx8q\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.597278 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.597215    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.597278 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.597256    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.597512 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.597294    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.597512 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.597348    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.598226 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.598198    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.599937 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.599890    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.600065 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.600038    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.600065 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.600058    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.600172 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.600104    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.610547 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.610520    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.612599 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.612574    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjx8q\" (UniqueName: \"kubernetes.io/projected/ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25-kube-api-access-fjx8q\") pod \"istiod-openshift-gateway-94bb8fbfd-sgkbf\" (UID: \"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.646640 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.646606    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:14.740235 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.740203    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:14.810802 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.810730    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:14.878729 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.878562    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"]
Jul 10 00:24:14.881197 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:24:14.881172    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddb4a0ba_d7b1_4d98_acf5_dca9abb9ea25.slice/crio-13cbff40d2dc103b64bc214178f12a04968a8e81f0b3983884479665df99685d WatchSource:0}: Error finding container 13cbff40d2dc103b64bc214178f12a04968a8e81f0b3983884479665df99685d: Status 404 returned error can't find the container with id 13cbff40d2dc103b64bc214178f12a04968a8e81f0b3983884479665df99685d
Jul 10 00:24:14.916432 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:14.916403    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:15.772456 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:15.772418    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf" event={"ID":"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25","Type":"ContainerStarted","Data":"13cbff40d2dc103b64bc214178f12a04968a8e81f0b3983884479665df99685d"}
Jul 10 00:24:16.471484 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.471448    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"]
Jul 10 00:24:16.475042 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.474999    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"
Jul 10 00:24:16.479746 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.479503    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-wgrz8\""
Jul 10 00:24:16.479746 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.479546    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 10 00:24:16.479746 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.479571    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 10 00:24:16.479746 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.479626    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 10 00:24:16.479746 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.479511    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 10 00:24:16.479746 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.479573    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 10 00:24:16.490257 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.490233    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"]
Jul 10 00:24:16.615789 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.615749    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3a76c30-d768-4393-935e-7eb2240fc615-metrics-cert\") pod \"lws-controller-manager-7b4d8999df-lzz45\" (UID: \"a3a76c30-d768-4393-935e-7eb2240fc615\") " pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"
Jul 10 00:24:16.616004 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.615802    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a3a76c30-d768-4393-935e-7eb2240fc615-manager-config\") pod \"lws-controller-manager-7b4d8999df-lzz45\" (UID: \"a3a76c30-d768-4393-935e-7eb2240fc615\") " pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"
Jul 10 00:24:16.616004 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.615865    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fndk2\" (UniqueName: \"kubernetes.io/projected/a3a76c30-d768-4393-935e-7eb2240fc615-kube-api-access-fndk2\") pod \"lws-controller-manager-7b4d8999df-lzz45\" (UID: \"a3a76c30-d768-4393-935e-7eb2240fc615\") " pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"
Jul 10 00:24:16.616004 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.615913    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3a76c30-d768-4393-935e-7eb2240fc615-cert\") pod \"lws-controller-manager-7b4d8999df-lzz45\" (UID: \"a3a76c30-d768-4393-935e-7eb2240fc615\") " pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"
Jul 10 00:24:16.717131 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.717096    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3a76c30-d768-4393-935e-7eb2240fc615-metrics-cert\") pod \"lws-controller-manager-7b4d8999df-lzz45\" (UID: \"a3a76c30-d768-4393-935e-7eb2240fc615\") " pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"
Jul 10 00:24:16.717131 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.717136    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a3a76c30-d768-4393-935e-7eb2240fc615-manager-config\") pod \"lws-controller-manager-7b4d8999df-lzz45\" (UID: \"a3a76c30-d768-4393-935e-7eb2240fc615\") " pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"
Jul 10 00:24:16.717439 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.717413    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fndk2\" (UniqueName: \"kubernetes.io/projected/a3a76c30-d768-4393-935e-7eb2240fc615-kube-api-access-fndk2\") pod \"lws-controller-manager-7b4d8999df-lzz45\" (UID: \"a3a76c30-d768-4393-935e-7eb2240fc615\") " pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"
Jul 10 00:24:16.717499 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.717475    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3a76c30-d768-4393-935e-7eb2240fc615-cert\") pod \"lws-controller-manager-7b4d8999df-lzz45\" (UID: \"a3a76c30-d768-4393-935e-7eb2240fc615\") " pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"
Jul 10 00:24:16.717868 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.717836    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a3a76c30-d768-4393-935e-7eb2240fc615-manager-config\") pod \"lws-controller-manager-7b4d8999df-lzz45\" (UID: \"a3a76c30-d768-4393-935e-7eb2240fc615\") " pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"
Jul 10 00:24:16.719994 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.719974    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3a76c30-d768-4393-935e-7eb2240fc615-metrics-cert\") pod \"lws-controller-manager-7b4d8999df-lzz45\" (UID: \"a3a76c30-d768-4393-935e-7eb2240fc615\") " pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"
Jul 10 00:24:16.720194 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.720165    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3a76c30-d768-4393-935e-7eb2240fc615-cert\") pod \"lws-controller-manager-7b4d8999df-lzz45\" (UID: \"a3a76c30-d768-4393-935e-7eb2240fc615\") " pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"
Jul 10 00:24:16.732806 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.732742    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fndk2\" (UniqueName: \"kubernetes.io/projected/a3a76c30-d768-4393-935e-7eb2240fc615-kube-api-access-fndk2\") pod \"lws-controller-manager-7b4d8999df-lzz45\" (UID: \"a3a76c30-d768-4393-935e-7eb2240fc615\") " pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"
Jul 10 00:24:16.790230 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:16.790189    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"
Jul 10 00:24:17.067587 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:17.067428    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:17.176045 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:17.175965    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:17.566193 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:17.566143    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 00:24:17.566294 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:17.566229    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 00:24:17.568529 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:17.568506    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:17.645821 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:17.645795    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"]
Jul 10 00:24:17.647914 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:24:17.647892    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3a76c30_d768_4393_935e_7eb2240fc615.slice/crio-09e09a7af82a160ab8325430d735cfa4ee69c851879198f205a98e2bf4186d25 WatchSource:0}: Error finding container 09e09a7af82a160ab8325430d735cfa4ee69c851879198f205a98e2bf4186d25: Status 404 returned error can't find the container with id 09e09a7af82a160ab8325430d735cfa4ee69c851879198f205a98e2bf4186d25
Jul 10 00:24:17.782567 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:17.782472    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45" event={"ID":"a3a76c30-d768-4393-935e-7eb2240fc615","Type":"ContainerStarted","Data":"09e09a7af82a160ab8325430d735cfa4ee69c851879198f205a98e2bf4186d25"}
Jul 10 00:24:17.784116 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:17.784085    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf" event={"ID":"ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25","Type":"ContainerStarted","Data":"fe0f1c911160c154860e0fb32051b074627bf8135086c97bdfce58e3dccdb755"}
Jul 10 00:24:17.784339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:17.784306    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:17.818820 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:17.818767    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf" podStartSLOduration=1.1361519549999999 podStartE2EDuration="3.818747727s" totalImagesPullingTime="2.682595772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:24:17.817395668 +0000 UTC m=+507.520180537" watchObservedRunningTime="2026-07-10 00:24:17.818747727 +0000 UTC m=+507.521532575"
Jul 10 00:24:18.450477 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:18.450443    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:18.543473 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:18.543436    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:18.789774 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:18.789689    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sgkbf"
Jul 10 00:24:21.757449 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:21.757412    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-snjhz"
Jul 10 00:24:21.804996 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:21.804964    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45" event={"ID":"a3a76c30-d768-4393-935e-7eb2240fc615","Type":"ContainerStarted","Data":"7cbc5ed4ccb6a62dacde00491c95f3e777976397624253651754c8ea1590c564"}
Jul 10 00:24:21.805153 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:21.805052    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"
Jul 10 00:24:21.828716 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:21.828663    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45" podStartSLOduration=2.088260789 podStartE2EDuration="5.828647302s" totalImagesPullingTime="3.740386513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:24:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:24:21.827169713 +0000 UTC m=+511.529954560" watchObservedRunningTime="2026-07-10 00:24:21.828647302 +0000 UTC m=+511.531432150"
Jul 10 00:24:32.811906 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:32.811874    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b4d8999df-lzz45"
Jul 10 00:24:40.226478 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:40.226442    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:40.275113 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:40.275079    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:42.176157 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:42.176063    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:24:42.231192 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:24:42.231158    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:08.285554 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:08.285495    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:08.349221 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:08.349187    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:08.407271 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:08.407235    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:09.113241 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:09.113210    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:09.311061 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:09.311027    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:27.645240 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:27.645204    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:30.776749 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:30.776715    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:30.993902 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:30.993870    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:33.771498 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:33.771466    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:34.578912 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:34.578879    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:36.594695 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:36.594656    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:37.840240 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:37.840209    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:38.053549 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:38.053511    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:38.159016 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:38.158938    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:38.265550 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:38.265517    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:38.560230 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:38.560200    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:38.662109 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:38.662074    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:40.603175 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:40.603149    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:43.592565 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:43.592528    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:43.699516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:43.699481    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:46.982439 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:46.982407    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:48.236264 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:48.236203    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:48.563268 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:48.563166    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:48.663587 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:48.663550    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:48.731295 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:48.731264    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:50.721276 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:50.721245    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:25:50.722414 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:50.722392    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:25:52.382212 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:52.382181    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:52.482350 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:52.482298    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:52.673134 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:52.673055    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:52.974525 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:52.974490    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:53.035686 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:53.035651    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:53.204796 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:53.204767    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:55.259140 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:55.259105    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:55.343121 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:55.343093    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:57.044453 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:57.044421    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:25:57.144667 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:25:57.144633    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:01.988045 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:01.988008    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:02.101888 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:02.101858    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:05.078013 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:05.077977    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:05.174795 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:05.174764    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:09.668642 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:09.668609    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:09.730587 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:09.730556    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:09.864569 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:09.864535    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:09.928541 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:09.928466    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:14.657661 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:14.657630    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:14.713495 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:14.713466    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:15.806489 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:15.806451    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"]
Jul 10 00:26:15.809772 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:15.809755    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"
Jul 10 00:26:15.812778 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:15.812752    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 00:26:15.812931 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:15.812788    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 00:26:15.812931 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:15.812807    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vkrqw\""
Jul 10 00:26:15.822041 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:15.822017    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"]
Jul 10 00:26:15.947125 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:15.947070    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwg86\" (UniqueName: \"kubernetes.io/projected/6da06b0d-43b7-4dc0-ac34-cec72df8006f-kube-api-access-dwg86\") pod \"kuadrant-operator-controller-manager-86878c769f-qkj4d\" (UID: \"6da06b0d-43b7-4dc0-ac34-cec72df8006f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"
Jul 10 00:26:15.947125 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:15.947130    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6da06b0d-43b7-4dc0-ac34-cec72df8006f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qkj4d\" (UID: \"6da06b0d-43b7-4dc0-ac34-cec72df8006f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"
Jul 10 00:26:16.048568 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:16.048525    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwg86\" (UniqueName: \"kubernetes.io/projected/6da06b0d-43b7-4dc0-ac34-cec72df8006f-kube-api-access-dwg86\") pod \"kuadrant-operator-controller-manager-86878c769f-qkj4d\" (UID: \"6da06b0d-43b7-4dc0-ac34-cec72df8006f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"
Jul 10 00:26:16.048748 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:16.048572    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6da06b0d-43b7-4dc0-ac34-cec72df8006f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qkj4d\" (UID: \"6da06b0d-43b7-4dc0-ac34-cec72df8006f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"
Jul 10 00:26:16.048977 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:16.048956    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6da06b0d-43b7-4dc0-ac34-cec72df8006f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qkj4d\" (UID: \"6da06b0d-43b7-4dc0-ac34-cec72df8006f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"
Jul 10 00:26:16.062610 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:16.062553    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwg86\" (UniqueName: \"kubernetes.io/projected/6da06b0d-43b7-4dc0-ac34-cec72df8006f-kube-api-access-dwg86\") pod \"kuadrant-operator-controller-manager-86878c769f-qkj4d\" (UID: \"6da06b0d-43b7-4dc0-ac34-cec72df8006f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"
Jul 10 00:26:16.120519 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:16.120487    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"
Jul 10 00:26:16.254560 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:16.254536    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"]
Jul 10 00:26:16.256973 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:26:16.256943    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da06b0d_43b7_4dc0_ac34_cec72df8006f.slice/crio-d6ed784cf89c2b267972454cbbf26170ff3e5fbee38922da7f1b38ea62e0fbf8 WatchSource:0}: Error finding container d6ed784cf89c2b267972454cbbf26170ff3e5fbee38922da7f1b38ea62e0fbf8: Status 404 returned error can't find the container with id d6ed784cf89c2b267972454cbbf26170ff3e5fbee38922da7f1b38ea62e0fbf8
Jul 10 00:26:16.288607 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:16.288580    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d" event={"ID":"6da06b0d-43b7-4dc0-ac34-cec72df8006f","Type":"ContainerStarted","Data":"d6ed784cf89c2b267972454cbbf26170ff3e5fbee38922da7f1b38ea62e0fbf8"}
Jul 10 00:26:23.323916 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:23.323878    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d" event={"ID":"6da06b0d-43b7-4dc0-ac34-cec72df8006f","Type":"ContainerStarted","Data":"140633718fc788aef476d3c6689a1fb9fbcace5b27cedbf3224ce1c3935dc1ac"}
Jul 10 00:26:23.324341 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:23.324003    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"
Jul 10 00:26:23.346450 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:23.346393    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d" podStartSLOduration=2.1318076 podStartE2EDuration="8.346376145s" totalImagesPullingTime="6.214568545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:26:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:23.344815048 +0000 UTC m=+633.047599915" watchObservedRunningTime="2026-07-10 00:26:23.346376145 +0000 UTC m=+633.049160993"
Jul 10 00:26:32.081222 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:32.081182    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:32.291973 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:32.291941    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:34.330208 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:34.330176    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"
Jul 10 00:26:54.534631 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:54.534589    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:54.637780 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:54.637745    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:54.687955 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:54.687924    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:55.350868 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:55.350834    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:26:55.452162 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:26:55.452090    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:27:07.137622 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:27:07.137564    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:27:16.434339 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:27:16.434283    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:27:16.533516 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:27:16.533479    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:27:16.948043 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:27:16.948015    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:27:27.597202 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:27:27.597168    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:27:39.382747 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:27:39.382713    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:27:39.482728 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:27:39.482687    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:27:45.511850 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:27:45.511810    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:27:45.570303 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:27:45.570269    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:27:45.625404 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:27:45.625372    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:27:45.820804 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:27:45.820719    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:27:48.829086 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:27:48.829037    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:27:48.928111 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:27:48.928073    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:28:01.201624 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:01.201583    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:28:01.255514 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:01.255477    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:28:23.540943 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:23.540907    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:28:24.030155 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.030120    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:28:24.052658 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.052629    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"]
Jul 10 00:28:24.055861 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.055841    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"
Jul 10 00:28:24.058429 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.058405    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 00:28:24.058544 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.058447    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 10 00:28:24.058544 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.058495    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 00:28:24.064543 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.064523    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"]
Jul 10 00:28:24.170664 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.170627    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3673ec78-c8fc-4b38-a892-be81d2738ad2-maas-api-tls\") pod \"maas-api-7bc864879c-27nt2\" (UID: \"3673ec78-c8fc-4b38-a892-be81d2738ad2\") " pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"
Jul 10 00:28:24.170861 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.170699    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjz5n\" (UniqueName: \"kubernetes.io/projected/3673ec78-c8fc-4b38-a892-be81d2738ad2-kube-api-access-wjz5n\") pod \"maas-api-7bc864879c-27nt2\" (UID: \"3673ec78-c8fc-4b38-a892-be81d2738ad2\") " pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"
Jul 10 00:28:24.170861 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.170733    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3673ec78-c8fc-4b38-a892-be81d2738ad2-openshift-service-ca\") pod \"maas-api-7bc864879c-27nt2\" (UID: \"3673ec78-c8fc-4b38-a892-be81d2738ad2\") " pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"
Jul 10 00:28:24.271887 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.271848    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjz5n\" (UniqueName: \"kubernetes.io/projected/3673ec78-c8fc-4b38-a892-be81d2738ad2-kube-api-access-wjz5n\") pod \"maas-api-7bc864879c-27nt2\" (UID: \"3673ec78-c8fc-4b38-a892-be81d2738ad2\") " pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"
Jul 10 00:28:24.272035 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.271913    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3673ec78-c8fc-4b38-a892-be81d2738ad2-openshift-service-ca\") pod \"maas-api-7bc864879c-27nt2\" (UID: \"3673ec78-c8fc-4b38-a892-be81d2738ad2\") " pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"
Jul 10 00:28:24.272035 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.271952    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3673ec78-c8fc-4b38-a892-be81d2738ad2-maas-api-tls\") pod \"maas-api-7bc864879c-27nt2\" (UID: \"3673ec78-c8fc-4b38-a892-be81d2738ad2\") " pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"
Jul 10 00:28:24.272138 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:28:24.272076    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 10 00:28:24.272176 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:28:24.272140    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3673ec78-c8fc-4b38-a892-be81d2738ad2-maas-api-tls podName:3673ec78-c8fc-4b38-a892-be81d2738ad2 nodeName:}" failed. No retries permitted until 2026-07-10 00:28:24.772119705 +0000 UTC m=+754.474904532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3673ec78-c8fc-4b38-a892-be81d2738ad2-maas-api-tls") pod "maas-api-7bc864879c-27nt2" (UID: "3673ec78-c8fc-4b38-a892-be81d2738ad2") : secret "maas-api-serving-cert" not found
Jul 10 00:28:24.272579 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.272561    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3673ec78-c8fc-4b38-a892-be81d2738ad2-openshift-service-ca\") pod \"maas-api-7bc864879c-27nt2\" (UID: \"3673ec78-c8fc-4b38-a892-be81d2738ad2\") " pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"
Jul 10 00:28:24.280588 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.280530    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjz5n\" (UniqueName: \"kubernetes.io/projected/3673ec78-c8fc-4b38-a892-be81d2738ad2-kube-api-access-wjz5n\") pod \"maas-api-7bc864879c-27nt2\" (UID: \"3673ec78-c8fc-4b38-a892-be81d2738ad2\") " pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"
Jul 10 00:28:24.450909 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.450875    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:28:24.547959 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.547855    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:28:24.549931 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.549906    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-w2zng"]
Jul 10 00:28:24.553388 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.553370    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-w2zng"
Jul 10 00:28:24.557101 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.557078    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-5xp79\""
Jul 10 00:28:24.557292 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.557275    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 10 00:28:24.564087 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.564065    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-w2zng"]
Jul 10 00:28:24.675607 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.675573    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh47k\" (UniqueName: \"kubernetes.io/projected/f577c229-6648-4358-8cde-ab4bcf0a4ae9-kube-api-access-mh47k\") pod \"payload-pre-processing-7964457696-w2zng\" (UID: \"f577c229-6648-4358-8cde-ab4bcf0a4ae9\") " pod="openshift-ingress/payload-pre-processing-7964457696-w2zng"
Jul 10 00:28:24.675758 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.675636    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f577c229-6648-4358-8cde-ab4bcf0a4ae9-plugins-config-volume\") pod \"payload-pre-processing-7964457696-w2zng\" (UID: \"f577c229-6648-4358-8cde-ab4bcf0a4ae9\") " pod="openshift-ingress/payload-pre-processing-7964457696-w2zng"
Jul 10 00:28:24.681043 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.681019    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:28:24.776259 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.776228    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3673ec78-c8fc-4b38-a892-be81d2738ad2-maas-api-tls\") pod \"maas-api-7bc864879c-27nt2\" (UID: \"3673ec78-c8fc-4b38-a892-be81d2738ad2\") " pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"
Jul 10 00:28:24.776455 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.776277    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh47k\" (UniqueName: \"kubernetes.io/projected/f577c229-6648-4358-8cde-ab4bcf0a4ae9-kube-api-access-mh47k\") pod \"payload-pre-processing-7964457696-w2zng\" (UID: \"f577c229-6648-4358-8cde-ab4bcf0a4ae9\") " pod="openshift-ingress/payload-pre-processing-7964457696-w2zng"
Jul 10 00:28:24.776455 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.776359    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f577c229-6648-4358-8cde-ab4bcf0a4ae9-plugins-config-volume\") pod \"payload-pre-processing-7964457696-w2zng\" (UID: \"f577c229-6648-4358-8cde-ab4bcf0a4ae9\") " pod="openshift-ingress/payload-pre-processing-7964457696-w2zng"
Jul 10 00:28:24.777038 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.777013    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f577c229-6648-4358-8cde-ab4bcf0a4ae9-plugins-config-volume\") pod \"payload-pre-processing-7964457696-w2zng\" (UID: \"f577c229-6648-4358-8cde-ab4bcf0a4ae9\") " pod="openshift-ingress/payload-pre-processing-7964457696-w2zng"
Jul 10 00:28:24.778694 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.778673    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3673ec78-c8fc-4b38-a892-be81d2738ad2-maas-api-tls\") pod \"maas-api-7bc864879c-27nt2\" (UID: \"3673ec78-c8fc-4b38-a892-be81d2738ad2\") " pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"
Jul 10 00:28:24.783709 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.783688    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh47k\" (UniqueName: \"kubernetes.io/projected/f577c229-6648-4358-8cde-ab4bcf0a4ae9-kube-api-access-mh47k\") pod \"payload-pre-processing-7964457696-w2zng\" (UID: \"f577c229-6648-4358-8cde-ab4bcf0a4ae9\") " pod="openshift-ingress/payload-pre-processing-7964457696-w2zng"
Jul 10 00:28:24.865379 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.865280    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-w2zng"
Jul 10 00:28:24.967662 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.967629    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"
Jul 10 00:28:24.993849 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:24.993822    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-w2zng"]
Jul 10 00:28:24.995639 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:28:24.995612    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf577c229_6648_4358_8cde_ab4bcf0a4ae9.slice/crio-96f03172571cb7d13250fbd42a179ffb195ea844891d228734387ed0a70ac87d WatchSource:0}: Error finding container 96f03172571cb7d13250fbd42a179ffb195ea844891d228734387ed0a70ac87d: Status 404 returned error can't find the container with id 96f03172571cb7d13250fbd42a179ffb195ea844891d228734387ed0a70ac87d
Jul 10 00:28:25.101876 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:25.101848    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"]
Jul 10 00:28:25.104301 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:28:25.104271    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3673ec78_c8fc_4b38_a892_be81d2738ad2.slice/crio-e144d4efd84ace00c42828dc2d1cbe94963503c249d5da8c03ff34102d4566ff WatchSource:0}: Error finding container e144d4efd84ace00c42828dc2d1cbe94963503c249d5da8c03ff34102d4566ff: Status 404 returned error can't find the container with id e144d4efd84ace00c42828dc2d1cbe94963503c249d5da8c03ff34102d4566ff
Jul 10 00:28:25.855677 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:25.855639    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2" event={"ID":"3673ec78-c8fc-4b38-a892-be81d2738ad2","Type":"ContainerStarted","Data":"e144d4efd84ace00c42828dc2d1cbe94963503c249d5da8c03ff34102d4566ff"}
Jul 10 00:28:25.857174 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:25.857144    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-w2zng" event={"ID":"f577c229-6648-4358-8cde-ab4bcf0a4ae9","Type":"ContainerStarted","Data":"96f03172571cb7d13250fbd42a179ffb195ea844891d228734387ed0a70ac87d"}
Jul 10 00:28:27.868538 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:27.868442    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2" event={"ID":"3673ec78-c8fc-4b38-a892-be81d2738ad2","Type":"ContainerStarted","Data":"9f05d1c7f4d5c1de1584a86a07221a166e179b31825a5af212a214928dca4782"}
Jul 10 00:28:27.868989 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:27.868564    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"
Jul 10 00:28:27.870072 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:27.870046    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-w2zng" event={"ID":"f577c229-6648-4358-8cde-ab4bcf0a4ae9","Type":"ContainerStarted","Data":"fae1179c6162e84a017a40d3e6db6b2c9c0b04e590ee9b8aea2a25fb16ff8695"}
Jul 10 00:28:27.870201 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:27.870142    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-w2zng"
Jul 10 00:28:27.886891 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:27.886847    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2" podStartSLOduration=1.415923511 podStartE2EDuration="3.886832658s" totalImagesPullingTime="2.470909147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:28:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:28:27.884415567 +0000 UTC m=+757.587200417" watchObservedRunningTime="2026-07-10 00:28:27.886832658 +0000 UTC m=+757.589617505"
Jul 10 00:28:27.901460 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:27.901407    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-w2zng" podStartSLOduration=1.369691907 podStartE2EDuration="3.901393549s" totalImagesPullingTime="2.531701642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:28:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:28:27.898507821 +0000 UTC m=+757.601292722" watchObservedRunningTime="2026-07-10 00:28:27.901393549 +0000 UTC m=+757.604178397"
Jul 10 00:28:31.360055 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:31.359995    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:28:33.880707 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:33.880681    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7bc864879c-27nt2"
Jul 10 00:28:34.938208 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:34.938178    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:28:35.038325 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:35.038279    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:28:38.877217 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:38.877189    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-w2zng"
Jul 10 00:28:48.725418 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:48.725384    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:28:56.784767 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.784724    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"]
Jul 10 00:28:56.788422 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.788398    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.791185 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.791164    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 10 00:28:56.792517 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.792495    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nzpmf\""
Jul 10 00:28:56.792632 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.792504    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 10 00:28:56.792632 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.792505    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 10 00:28:56.797848 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.797828    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"]
Jul 10 00:28:56.856196 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.856161    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.856355 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.856216    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.856355 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.856239    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxqlf\" (UniqueName: \"kubernetes.io/projected/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-kube-api-access-rxqlf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.856355 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.856258    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.856355 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.856284    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.856507 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.856378    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.895329 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.895285    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:28:56.957274 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.957235    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.957496 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.957293    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.957496 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.957381    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.957496 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.957401    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxqlf\" (UniqueName: \"kubernetes.io/projected/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-kube-api-access-rxqlf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.957496 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.957432    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.957496 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.957464    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.957770 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.957742    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.957770 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.957766    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.957911 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.957892    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.959646 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.959617    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.959882 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.959864    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:56.965616 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:56.965592    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxqlf\" (UniqueName: \"kubernetes.io/projected/e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f-kube-api-access-rxqlf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4wkgc\" (UID: \"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:57.099478 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:57.099395    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:28:57.230982 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:57.230954    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"]
Jul 10 00:28:57.232800 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:28:57.232774    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8a42e84_5cc0_4355_aeb4_a92a5f0cc41f.slice/crio-c45e830c226ad8dcf41ed552b873a19d3f6118b0cd1562eadd614c3689f54291 WatchSource:0}: Error finding container c45e830c226ad8dcf41ed552b873a19d3f6118b0cd1562eadd614c3689f54291: Status 404 returned error can't find the container with id c45e830c226ad8dcf41ed552b873a19d3f6118b0cd1562eadd614c3689f54291
Jul 10 00:28:57.234546 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:57.234525    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 00:28:58.010164 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:58.010129    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc" event={"ID":"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f","Type":"ContainerStarted","Data":"c45e830c226ad8dcf41ed552b873a19d3f6118b0cd1562eadd614c3689f54291"}
Jul 10 00:28:59.772988 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:28:59.772954    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:29:06.048587 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:06.048542    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc" event={"ID":"e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f","Type":"ContainerStarted","Data":"7c27947fb194824f2915af08de848cb13c3a242e4cea1131bfc7e6cf6404b34b"}
Jul 10 00:29:06.066949 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:06.066893    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc" podStartSLOduration=1.870629181 podStartE2EDuration="10.066877118s" totalImagesPullingTime="8.196247937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:28:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:29:06.064980403 +0000 UTC m=+795.767765297" watchObservedRunningTime="2026-07-10 00:29:06.066877118 +0000 UTC m=+795.769661966"
Jul 10 00:29:07.099637 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:07.099599    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:29:07.100083 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:07.099684    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:29:07.112001 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:07.111973    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:29:08.070936 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:08.070909    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4wkgc"
Jul 10 00:29:13.982637 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:13.982605    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"]
Jul 10 00:29:13.985460 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:13.985434    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:13.988033 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:13.988011    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 10 00:29:13.994353 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:13.994330    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"]
Jul 10 00:29:14.011830 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.011799    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:29:14.108280 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.108252    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a84e1655-76f8-45b7-b2f2-ed59a51e244c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.108513 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.108284    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hv79\" (UniqueName: \"kubernetes.io/projected/a84e1655-76f8-45b7-b2f2-ed59a51e244c-kube-api-access-8hv79\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.108513 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.108337    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a84e1655-76f8-45b7-b2f2-ed59a51e244c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.108513 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.108391    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a84e1655-76f8-45b7-b2f2-ed59a51e244c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.108513 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.108499    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a84e1655-76f8-45b7-b2f2-ed59a51e244c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.108674 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.108518    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a84e1655-76f8-45b7-b2f2-ed59a51e244c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.209203 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.209171    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a84e1655-76f8-45b7-b2f2-ed59a51e244c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.209203 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.209205    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a84e1655-76f8-45b7-b2f2-ed59a51e244c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.209488 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.209227    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a84e1655-76f8-45b7-b2f2-ed59a51e244c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.209488 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.209243    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hv79\" (UniqueName: \"kubernetes.io/projected/a84e1655-76f8-45b7-b2f2-ed59a51e244c-kube-api-access-8hv79\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.209488 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.209355    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a84e1655-76f8-45b7-b2f2-ed59a51e244c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.209488 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.209391    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a84e1655-76f8-45b7-b2f2-ed59a51e244c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.209722 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.209696    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a84e1655-76f8-45b7-b2f2-ed59a51e244c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.209826 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.209808    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a84e1655-76f8-45b7-b2f2-ed59a51e244c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.209881 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.209828    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a84e1655-76f8-45b7-b2f2-ed59a51e244c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.211604 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.211581    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a84e1655-76f8-45b7-b2f2-ed59a51e244c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.211845 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.211829    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a84e1655-76f8-45b7-b2f2-ed59a51e244c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.216971 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.216947    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hv79\" (UniqueName: \"kubernetes.io/projected/a84e1655-76f8-45b7-b2f2-ed59a51e244c-kube-api-access-8hv79\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hv5t7\" (UID: \"a84e1655-76f8-45b7-b2f2-ed59a51e244c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.297249 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.297166    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:14.425476 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:14.425449    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"]
Jul 10 00:29:14.427534 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:29:14.427508    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda84e1655_76f8_45b7_b2f2_ed59a51e244c.slice/crio-cc495f5c7f3c5fed5df589849bf3fc43d6beb1e7be03b34ce6cb6c9856d2fdbb WatchSource:0}: Error finding container cc495f5c7f3c5fed5df589849bf3fc43d6beb1e7be03b34ce6cb6c9856d2fdbb: Status 404 returned error can't find the container with id cc495f5c7f3c5fed5df589849bf3fc43d6beb1e7be03b34ce6cb6c9856d2fdbb
Jul 10 00:29:15.093650 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:15.093604    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7" event={"ID":"a84e1655-76f8-45b7-b2f2-ed59a51e244c","Type":"ContainerStarted","Data":"9ccd8d6813d5157e32c069b8217f4392ffa8c0ab39b09022342b6d96e2009193"}
Jul 10 00:29:15.094063 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:15.093667    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7" event={"ID":"a84e1655-76f8-45b7-b2f2-ed59a51e244c","Type":"ContainerStarted","Data":"cc495f5c7f3c5fed5df589849bf3fc43d6beb1e7be03b34ce6cb6c9856d2fdbb"}
Jul 10 00:29:15.112806 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:15.112747    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7" podStartSLOduration=1.962742591 podStartE2EDuration="2.112726992s" totalImagesPullingTime="149.984401ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:29:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:29:15.110540936 +0000 UTC m=+804.813325808" watchObservedRunningTime="2026-07-10 00:29:15.112726992 +0000 UTC m=+804.815511841"
Jul 10 00:29:24.298222 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:24.298179    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:24.298795 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:24.298246    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:24.311101 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:24.311068    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:25.019576 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:25.019541    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:29:25.151952 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:25.151922    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hv5t7"
Jul 10 00:29:36.420225 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:36.420192    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:29:43.109137 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:29:43.109100    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:30:00.140628 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:00.140595    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727390-mfbtf"]
Jul 10 00:30:00.144265 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:00.144248    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727390-mfbtf"
Jul 10 00:30:00.146861 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:00.146841    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gfl2c\""
Jul 10 00:30:00.150714 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:00.150689    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727390-mfbtf"]
Jul 10 00:30:00.155354 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:00.155336    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727390-mfbtf"
Jul 10 00:30:00.285304 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:00.285279    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727390-mfbtf"]
Jul 10 00:30:00.287284 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:30:00.287256    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64cd853e_b500_4196_b46b_d23988a18a0a.slice/crio-0fe638d086b1493246002871a2e675aa846ae223fc94bddd7df831222248d7a5 WatchSource:0}: Error finding container 0fe638d086b1493246002871a2e675aa846ae223fc94bddd7df831222248d7a5: Status 404 returned error can't find the container with id 0fe638d086b1493246002871a2e675aa846ae223fc94bddd7df831222248d7a5
Jul 10 00:30:00.298499 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:00.298470    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727390-mfbtf" event={"ID":"64cd853e-b500-4196-b46b-d23988a18a0a","Type":"ContainerStarted","Data":"0fe638d086b1493246002871a2e675aa846ae223fc94bddd7df831222248d7a5"}
Jul 10 00:30:03.314304 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:03.314269    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727390-mfbtf" event={"ID":"64cd853e-b500-4196-b46b-d23988a18a0a","Type":"ContainerStarted","Data":"2ee85201122d2775b3b1d3d0a59ddbd1b7baf0160affdd9b236866d1ebdec60e"}
Jul 10 00:30:03.328930 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:03.328883    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727390-mfbtf" podStartSLOduration=1.118971819 podStartE2EDuration="3.328869054s" totalImagesPullingTime="2.209897235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:30:03.3280233 +0000 UTC m=+853.030808148" watchObservedRunningTime="2026-07-10 00:30:03.328869054 +0000 UTC m=+853.031653901"
Jul 10 00:30:43.495538 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:43.495506    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:30:43.597603 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:43.597565    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:30:47.284132 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:47.284094    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:30:47.416222 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:47.416186    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:30:47.513375 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:47.513331    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:30:47.613805 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:47.613716    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:30:48.487483 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:48.487452    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:30:49.581438 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:49.581401    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:30:50.749765 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:50.749736    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:30:50.750751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:30:50.750729    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:31:00.015157 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:31:00.015122    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727390-mfbtf"]
Jul 10 00:31:00.016634 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:31:00.015309    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727390-mfbtf" podUID="64cd853e-b500-4196-b46b-d23988a18a0a" containerName="cleanup" containerID="cri-o://2ee85201122d2775b3b1d3d0a59ddbd1b7baf0160affdd9b236866d1ebdec60e" gracePeriod=30
Jul 10 00:31:01.429798 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:31:01.429762    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:31:30.191081 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:31:30.191054    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727390-mfbtf"
Jul 10 00:31:30.704230 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:31:30.704192    2559 generic.go:356] "Generic (PLEG): container finished" podID="64cd853e-b500-4196-b46b-d23988a18a0a" containerID="2ee85201122d2775b3b1d3d0a59ddbd1b7baf0160affdd9b236866d1ebdec60e" exitCode=137
Jul 10 00:31:30.704452 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:31:30.704240    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727390-mfbtf"
Jul 10 00:31:30.704452 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:31:30.704271    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727390-mfbtf" event={"ID":"64cd853e-b500-4196-b46b-d23988a18a0a","Type":"ContainerDied","Data":"2ee85201122d2775b3b1d3d0a59ddbd1b7baf0160affdd9b236866d1ebdec60e"}
Jul 10 00:31:30.704452 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:31:30.704335    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727390-mfbtf" event={"ID":"64cd853e-b500-4196-b46b-d23988a18a0a","Type":"ContainerDied","Data":"0fe638d086b1493246002871a2e675aa846ae223fc94bddd7df831222248d7a5"}
Jul 10 00:31:30.704452 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:31:30.704363    2559 scope.go:117] "RemoveContainer" containerID="2ee85201122d2775b3b1d3d0a59ddbd1b7baf0160affdd9b236866d1ebdec60e"
Jul 10 00:31:30.712986 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:31:30.712967    2559 scope.go:117] "RemoveContainer" containerID="2ee85201122d2775b3b1d3d0a59ddbd1b7baf0160affdd9b236866d1ebdec60e"
Jul 10 00:31:30.713286 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:31:30.713266    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ee85201122d2775b3b1d3d0a59ddbd1b7baf0160affdd9b236866d1ebdec60e\": container with ID starting with 2ee85201122d2775b3b1d3d0a59ddbd1b7baf0160affdd9b236866d1ebdec60e not found: ID does not exist" containerID="2ee85201122d2775b3b1d3d0a59ddbd1b7baf0160affdd9b236866d1ebdec60e"
Jul 10 00:31:30.713386 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:31:30.713295    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ee85201122d2775b3b1d3d0a59ddbd1b7baf0160affdd9b236866d1ebdec60e"} err="failed to get container status \"2ee85201122d2775b3b1d3d0a59ddbd1b7baf0160affdd9b236866d1ebdec60e\": rpc error: code = NotFound desc = could not find container \"2ee85201122d2775b3b1d3d0a59ddbd1b7baf0160affdd9b236866d1ebdec60e\": container with ID starting with 2ee85201122d2775b3b1d3d0a59ddbd1b7baf0160affdd9b236866d1ebdec60e not found: ID does not exist"
Jul 10 00:31:30.724949 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:31:30.724920    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727390-mfbtf"]
Jul 10 00:31:30.728870 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:31:30.728846    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727390-mfbtf"]
Jul 10 00:31:30.837122 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:31:30.837094    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64cd853e-b500-4196-b46b-d23988a18a0a" path="/var/lib/kubelet/pods/64cd853e-b500-4196-b46b-d23988a18a0a/volumes"
Jul 10 00:31:59.576786 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:31:59.576749    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:32:01.015191 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:32:01.015158    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:32:46.387470 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:32:46.387419    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:32:46.442978 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:32:46.442949    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:32:51.901612 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:32:51.901567    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:32:51.964521 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:32:51.964490    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:32:52.038773 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:32:52.038739    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:32:52.137520 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:32:52.137486    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:32:52.325903 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:32:52.325866    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:32:57.859198 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:32:57.859161    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:32:57.959220 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:32:57.959183    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:34:32.658124 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:34:32.658088    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:34:32.708148 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:34:32.708114    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:35:08.632829 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:35:08.632788    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:35:50.777661 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:35:50.777626    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:35:50.780031 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:35:50.780008    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:36:50.693302 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:50.693214    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"]
Jul 10 00:36:50.695918 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:50.693480    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d" podUID="6da06b0d-43b7-4dc0-ac34-cec72df8006f" containerName="manager" containerID="cri-o://140633718fc788aef476d3c6689a1fb9fbcace5b27cedbf3224ce1c3935dc1ac" gracePeriod=10
Jul 10 00:36:54.198019 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:54.197983    2559 generic.go:356] "Generic (PLEG): container finished" podID="6da06b0d-43b7-4dc0-ac34-cec72df8006f" containerID="140633718fc788aef476d3c6689a1fb9fbcace5b27cedbf3224ce1c3935dc1ac" exitCode=0
Jul 10 00:36:54.198448 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:54.198045    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d" event={"ID":"6da06b0d-43b7-4dc0-ac34-cec72df8006f","Type":"ContainerDied","Data":"140633718fc788aef476d3c6689a1fb9fbcace5b27cedbf3224ce1c3935dc1ac"}
Jul 10 00:36:54.244185 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:54.244158    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"
Jul 10 00:36:54.295639 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:54.295604    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6da06b0d-43b7-4dc0-ac34-cec72df8006f-extensions-socket-volume\") pod \"6da06b0d-43b7-4dc0-ac34-cec72df8006f\" (UID: \"6da06b0d-43b7-4dc0-ac34-cec72df8006f\") "
Jul 10 00:36:54.295824 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:54.295657    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwg86\" (UniqueName: \"kubernetes.io/projected/6da06b0d-43b7-4dc0-ac34-cec72df8006f-kube-api-access-dwg86\") pod \"6da06b0d-43b7-4dc0-ac34-cec72df8006f\" (UID: \"6da06b0d-43b7-4dc0-ac34-cec72df8006f\") "
Jul 10 00:36:54.296023 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:54.295996    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6da06b0d-43b7-4dc0-ac34-cec72df8006f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6da06b0d-43b7-4dc0-ac34-cec72df8006f" (UID: "6da06b0d-43b7-4dc0-ac34-cec72df8006f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 00:36:54.297812 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:54.297780    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6da06b0d-43b7-4dc0-ac34-cec72df8006f-kube-api-access-dwg86" (OuterVolumeSpecName: "kube-api-access-dwg86") pod "6da06b0d-43b7-4dc0-ac34-cec72df8006f" (UID: "6da06b0d-43b7-4dc0-ac34-cec72df8006f"). InnerVolumeSpecName "kube-api-access-dwg86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:36:54.396892 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:54.396809    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6da06b0d-43b7-4dc0-ac34-cec72df8006f-extensions-socket-volume\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:36:54.396892 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:54.396840    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwg86\" (UniqueName: \"kubernetes.io/projected/6da06b0d-43b7-4dc0-ac34-cec72df8006f-kube-api-access-dwg86\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:36:55.203588 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:55.203562    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"
Jul 10 00:36:55.203588 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:55.203581    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d" event={"ID":"6da06b0d-43b7-4dc0-ac34-cec72df8006f","Type":"ContainerDied","Data":"d6ed784cf89c2b267972454cbbf26170ff3e5fbee38922da7f1b38ea62e0fbf8"}
Jul 10 00:36:55.204060 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:55.203642    2559 scope.go:117] "RemoveContainer" containerID="140633718fc788aef476d3c6689a1fb9fbcace5b27cedbf3224ce1c3935dc1ac"
Jul 10 00:36:55.223179 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:55.223152    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"]
Jul 10 00:36:55.228679 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:55.228651    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qkj4d"]
Jul 10 00:36:56.836867 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:36:56.836836    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6da06b0d-43b7-4dc0-ac34-cec72df8006f" path="/var/lib/kubelet/pods/6da06b0d-43b7-4dc0-ac34-cec72df8006f/volumes"
Jul 10 00:37:04.539737 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:04.539704    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:37:51.858016 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.857981    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrmln"]
Jul 10 00:37:51.858508 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.858305    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6da06b0d-43b7-4dc0-ac34-cec72df8006f" containerName="manager"
Jul 10 00:37:51.858508 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.858329    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6da06b0d-43b7-4dc0-ac34-cec72df8006f" containerName="manager"
Jul 10 00:37:51.858508 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.858340    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64cd853e-b500-4196-b46b-d23988a18a0a" containerName="cleanup"
Jul 10 00:37:51.858508 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.858349    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="64cd853e-b500-4196-b46b-d23988a18a0a" containerName="cleanup"
Jul 10 00:37:51.858508 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.858422    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="64cd853e-b500-4196-b46b-d23988a18a0a" containerName="cleanup"
Jul 10 00:37:51.858508 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.858430    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6da06b0d-43b7-4dc0-ac34-cec72df8006f" containerName="manager"
Jul 10 00:37:51.861603 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.861586    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrmln"
Jul 10 00:37:51.864215 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.864192    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 00:37:51.864347 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.864276    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vkrqw\""
Jul 10 00:37:51.864590 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.864351    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 00:37:51.871669 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.871644    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrmln"]
Jul 10 00:37:51.887491 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.887458    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/405b6350-e566-49ab-81e0-59f70c86c6a3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rrmln\" (UID: \"405b6350-e566-49ab-81e0-59f70c86c6a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrmln"
Jul 10 00:37:51.887651 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.887498    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq587\" (UniqueName: \"kubernetes.io/projected/405b6350-e566-49ab-81e0-59f70c86c6a3-kube-api-access-pq587\") pod \"kuadrant-operator-controller-manager-86878c769f-rrmln\" (UID: \"405b6350-e566-49ab-81e0-59f70c86c6a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrmln"
Jul 10 00:37:51.989001 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.988908    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/405b6350-e566-49ab-81e0-59f70c86c6a3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rrmln\" (UID: \"405b6350-e566-49ab-81e0-59f70c86c6a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrmln"
Jul 10 00:37:51.989001 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.988971    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq587\" (UniqueName: \"kubernetes.io/projected/405b6350-e566-49ab-81e0-59f70c86c6a3-kube-api-access-pq587\") pod \"kuadrant-operator-controller-manager-86878c769f-rrmln\" (UID: \"405b6350-e566-49ab-81e0-59f70c86c6a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrmln"
Jul 10 00:37:51.989355 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.989335    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/405b6350-e566-49ab-81e0-59f70c86c6a3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rrmln\" (UID: \"405b6350-e566-49ab-81e0-59f70c86c6a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrmln"
Jul 10 00:37:51.997258 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:51.997235    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq587\" (UniqueName: \"kubernetes.io/projected/405b6350-e566-49ab-81e0-59f70c86c6a3-kube-api-access-pq587\") pod \"kuadrant-operator-controller-manager-86878c769f-rrmln\" (UID: \"405b6350-e566-49ab-81e0-59f70c86c6a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrmln"
Jul 10 00:37:52.173159 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:52.173120    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrmln"
Jul 10 00:37:52.304625 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:52.304601    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrmln"]
Jul 10 00:37:52.307391 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:37:52.307360    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod405b6350_e566_49ab_81e0_59f70c86c6a3.slice/crio-c1aa88c2eb43f293a0935e1ff3d7eca68da7a7232c8a4162ff7c0c2f7f651d51 WatchSource:0}: Error finding container c1aa88c2eb43f293a0935e1ff3d7eca68da7a7232c8a4162ff7c0c2f7f651d51: Status 404 returned error can't find the container with id c1aa88c2eb43f293a0935e1ff3d7eca68da7a7232c8a4162ff7c0c2f7f651d51
Jul 10 00:37:52.461416 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:52.461377    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrmln" event={"ID":"405b6350-e566-49ab-81e0-59f70c86c6a3","Type":"ContainerStarted","Data":"be3dea5ccf0315681f8e5cba699e90a6f98742aff4b1949f76d7d1fdb9f3440b"}
Jul 10 00:37:52.461620 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:52.461442    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrmln" event={"ID":"405b6350-e566-49ab-81e0-59f70c86c6a3","Type":"ContainerStarted","Data":"c1aa88c2eb43f293a0935e1ff3d7eca68da7a7232c8a4162ff7c0c2f7f651d51"}
Jul 10 00:37:52.461620 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:52.461511    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrmln"
Jul 10 00:37:52.480363 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:37:52.480293    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrmln" podStartSLOduration=1.480279517 podStartE2EDuration="1.480279517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:37:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:37:52.478735972 +0000 UTC m=+1322.181520819" watchObservedRunningTime="2026-07-10 00:37:52.480279517 +0000 UTC m=+1322.183064409"
Jul 10 00:38:03.469257 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:38:03.469228    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrmln"
Jul 10 00:40:23.443794 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:40:23.443749    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:40:23.500351 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:40:23.500268    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:40:50.809610 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:40:50.809582    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:40:50.811749 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:40:50.811723    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:43:35.109774 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:43:35.109736    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:43:42.605969 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:43:42.605929    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:43:46.645596 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:43:46.645554    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:43:46.746201 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:43:46.746170    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:43:50.899003 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:43:50.898965    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:43:56.026661 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:43:56.026623    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:43:59.010520 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:43:59.010480    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:43:59.110136 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:43:59.110097    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:43:59.466628 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:43:59.466592    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:05.149430 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:05.149394    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:05.209835 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:05.209799    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:05.349703 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:05.349663    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:05.410466 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:05.410343    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:06.449544 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:06.449512    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:06.549528 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:06.549491    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:06.606763 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:06.606724    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"]
Jul 10 00:44:06.610716 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:06.610689    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"
Jul 10 00:44:06.613470 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:06.613443    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-95438d51-serving-cert\""
Jul 10 00:44:06.613602 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:06.613477    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vpn7x\""
Jul 10 00:44:06.617837 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:06.617814    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"]
Jul 10 00:44:06.627203 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:06.627177    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4313d1f-2954-4935-a41c-9f9fab7ccf95-openshift-service-ca\") pod \"maas-api-e2e-ait-95438d51-5877d975f5-fj6t4\" (UID: \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"
Jul 10 00:44:06.627358 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:06.627209    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4313d1f-2954-4935-a41c-9f9fab7ccf95-maas-api-tls\") pod \"maas-api-e2e-ait-95438d51-5877d975f5-fj6t4\" (UID: \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"
Jul 10 00:44:06.627358 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:06.627236    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76gcn\" (UniqueName: \"kubernetes.io/projected/d4313d1f-2954-4935-a41c-9f9fab7ccf95-kube-api-access-76gcn\") pod \"maas-api-e2e-ait-95438d51-5877d975f5-fj6t4\" (UID: \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"
Jul 10 00:44:06.658351 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:06.658293    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:06.728380 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:06.728289    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4313d1f-2954-4935-a41c-9f9fab7ccf95-maas-api-tls\") pod \"maas-api-e2e-ait-95438d51-5877d975f5-fj6t4\" (UID: \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"
Jul 10 00:44:06.728380 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:06.728356    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76gcn\" (UniqueName: \"kubernetes.io/projected/d4313d1f-2954-4935-a41c-9f9fab7ccf95-kube-api-access-76gcn\") pod \"maas-api-e2e-ait-95438d51-5877d975f5-fj6t4\" (UID: \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"
Jul 10 00:44:06.728564 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:06.728474    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4313d1f-2954-4935-a41c-9f9fab7ccf95-openshift-service-ca\") pod \"maas-api-e2e-ait-95438d51-5877d975f5-fj6t4\" (UID: \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"
Jul 10 00:44:06.728564 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:44:06.728528    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-serving-cert: secret "maas-api-e2e-ait-95438d51-serving-cert" not found
Jul 10 00:44:06.728633 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:44:06.728611    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4313d1f-2954-4935-a41c-9f9fab7ccf95-maas-api-tls podName:d4313d1f-2954-4935-a41c-9f9fab7ccf95 nodeName:}" failed. No retries permitted until 2026-07-10 00:44:07.228589143 +0000 UTC m=+1696.931373974 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d4313d1f-2954-4935-a41c-9f9fab7ccf95-maas-api-tls") pod "maas-api-e2e-ait-95438d51-5877d975f5-fj6t4" (UID: "d4313d1f-2954-4935-a41c-9f9fab7ccf95") : secret "maas-api-e2e-ait-95438d51-serving-cert" not found
Jul 10 00:44:06.729101 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:06.729083    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4313d1f-2954-4935-a41c-9f9fab7ccf95-openshift-service-ca\") pod \"maas-api-e2e-ait-95438d51-5877d975f5-fj6t4\" (UID: \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"
Jul 10 00:44:06.738276 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:06.738251    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76gcn\" (UniqueName: \"kubernetes.io/projected/d4313d1f-2954-4935-a41c-9f9fab7ccf95-kube-api-access-76gcn\") pod \"maas-api-e2e-ait-95438d51-5877d975f5-fj6t4\" (UID: \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"
Jul 10 00:44:07.091889 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:07.091789    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:07.193875 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:07.193834    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:07.233047 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:07.233009    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4313d1f-2954-4935-a41c-9f9fab7ccf95-maas-api-tls\") pod \"maas-api-e2e-ait-95438d51-5877d975f5-fj6t4\" (UID: \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"
Jul 10 00:44:07.233224 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:44:07.233165    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-serving-cert: secret "maas-api-e2e-ait-95438d51-serving-cert" not found
Jul 10 00:44:07.233266 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:44:07.233240    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4313d1f-2954-4935-a41c-9f9fab7ccf95-maas-api-tls podName:d4313d1f-2954-4935-a41c-9f9fab7ccf95 nodeName:}" failed. No retries permitted until 2026-07-10 00:44:08.233224496 +0000 UTC m=+1697.936009322 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d4313d1f-2954-4935-a41c-9f9fab7ccf95-maas-api-tls") pod "maas-api-e2e-ait-95438d51-5877d975f5-fj6t4" (UID: "d4313d1f-2954-4935-a41c-9f9fab7ccf95") : secret "maas-api-e2e-ait-95438d51-serving-cert" not found
Jul 10 00:44:07.520899 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:07.520863    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:08.242724 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:08.242680    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4313d1f-2954-4935-a41c-9f9fab7ccf95-maas-api-tls\") pod \"maas-api-e2e-ait-95438d51-5877d975f5-fj6t4\" (UID: \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"
Jul 10 00:44:08.245231 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:08.245204    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4313d1f-2954-4935-a41c-9f9fab7ccf95-maas-api-tls\") pod \"maas-api-e2e-ait-95438d51-5877d975f5-fj6t4\" (UID: \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"
Jul 10 00:44:08.425034 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:08.424996    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"
Jul 10 00:44:08.556147 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:08.556101    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"]
Jul 10 00:44:08.558706 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:44:08.558680    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4313d1f_2954_4935_a41c_9f9fab7ccf95.slice/crio-0ecd1464606dfd2997d1b2fc96f7ef27f20dbcc1510ec0b5272d4b8e806a8a06 WatchSource:0}: Error finding container 0ecd1464606dfd2997d1b2fc96f7ef27f20dbcc1510ec0b5272d4b8e806a8a06: Status 404 returned error can't find the container with id 0ecd1464606dfd2997d1b2fc96f7ef27f20dbcc1510ec0b5272d4b8e806a8a06
Jul 10 00:44:08.560070 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:08.560050    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 00:44:09.068033 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:09.068004    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:09.164979 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:09.164941    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4" event={"ID":"d4313d1f-2954-4935-a41c-9f9fab7ccf95","Type":"ContainerStarted","Data":"aae280bfb923af115784d5194e0d85cb1b97dd8776d5f13d2ecbc4342e4e24c0"}
Jul 10 00:44:09.165160 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:09.164997    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4" event={"ID":"d4313d1f-2954-4935-a41c-9f9fab7ccf95","Type":"ContainerStarted","Data":"0ecd1464606dfd2997d1b2fc96f7ef27f20dbcc1510ec0b5272d4b8e806a8a06"}
Jul 10 00:44:09.165160 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:09.165018    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"
Jul 10 00:44:09.181299 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:09.181256    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4" podStartSLOduration=2.761658437 podStartE2EDuration="3.181241306s" totalImagesPullingTime="419.582869ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:44:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:44:09.180063098 +0000 UTC m=+1698.882847965" watchObservedRunningTime="2026-07-10 00:44:09.181241306 +0000 UTC m=+1698.884026154"
Jul 10 00:44:11.622685 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:11.622630    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:11.843085 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:11.843045    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:11.935155 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:11.935058    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:11.992839 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:11.992805    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"]
Jul 10 00:44:11.993071 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:11.993042    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4" podUID="d4313d1f-2954-4935-a41c-9f9fab7ccf95" containerName="maas-api" containerID="cri-o://aae280bfb923af115784d5194e0d85cb1b97dd8776d5f13d2ecbc4342e4e24c0" gracePeriod=30
Jul 10 00:44:12.016550 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:12.016519    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:12.187500 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:12.187407    2559 generic.go:356] "Generic (PLEG): container finished" podID="d4313d1f-2954-4935-a41c-9f9fab7ccf95" containerID="aae280bfb923af115784d5194e0d85cb1b97dd8776d5f13d2ecbc4342e4e24c0" exitCode=0
Jul 10 00:44:12.187659 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:12.187535    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4" event={"ID":"d4313d1f-2954-4935-a41c-9f9fab7ccf95","Type":"ContainerDied","Data":"aae280bfb923af115784d5194e0d85cb1b97dd8776d5f13d2ecbc4342e4e24c0"}
Jul 10 00:44:12.237585 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:12.237559    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"
Jul 10 00:44:12.279865 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:12.279834    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4313d1f-2954-4935-a41c-9f9fab7ccf95-openshift-service-ca\") pod \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\" (UID: \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\") "
Jul 10 00:44:12.280034 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:12.279907    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4313d1f-2954-4935-a41c-9f9fab7ccf95-maas-api-tls\") pod \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\" (UID: \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\") "
Jul 10 00:44:12.280034 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:12.279939    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76gcn\" (UniqueName: \"kubernetes.io/projected/d4313d1f-2954-4935-a41c-9f9fab7ccf95-kube-api-access-76gcn\") pod \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\" (UID: \"d4313d1f-2954-4935-a41c-9f9fab7ccf95\") "
Jul 10 00:44:12.280308 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:12.280285    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4313d1f-2954-4935-a41c-9f9fab7ccf95-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d4313d1f-2954-4935-a41c-9f9fab7ccf95" (UID: "d4313d1f-2954-4935-a41c-9f9fab7ccf95"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:44:12.282124 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:12.282102    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4313d1f-2954-4935-a41c-9f9fab7ccf95-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d4313d1f-2954-4935-a41c-9f9fab7ccf95" (UID: "d4313d1f-2954-4935-a41c-9f9fab7ccf95"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:44:12.282251 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:12.282230    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4313d1f-2954-4935-a41c-9f9fab7ccf95-kube-api-access-76gcn" (OuterVolumeSpecName: "kube-api-access-76gcn") pod "d4313d1f-2954-4935-a41c-9f9fab7ccf95" (UID: "d4313d1f-2954-4935-a41c-9f9fab7ccf95"). InnerVolumeSpecName "kube-api-access-76gcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:44:12.381440 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:12.381407    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4313d1f-2954-4935-a41c-9f9fab7ccf95-openshift-service-ca\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:44:12.381440 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:12.381437    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4313d1f-2954-4935-a41c-9f9fab7ccf95-maas-api-tls\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:44:12.381440 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:12.381449    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76gcn\" (UniqueName: \"kubernetes.io/projected/d4313d1f-2954-4935-a41c-9f9fab7ccf95-kube-api-access-76gcn\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:44:13.194194 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:13.194171    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"
Jul 10 00:44:13.194615 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:13.194220    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4" event={"ID":"d4313d1f-2954-4935-a41c-9f9fab7ccf95","Type":"ContainerDied","Data":"0ecd1464606dfd2997d1b2fc96f7ef27f20dbcc1510ec0b5272d4b8e806a8a06"}
Jul 10 00:44:13.194615 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:13.194284    2559 scope.go:117] "RemoveContainer" containerID="aae280bfb923af115784d5194e0d85cb1b97dd8776d5f13d2ecbc4342e4e24c0"
Jul 10 00:44:13.211241 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:13.211221    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"]
Jul 10 00:44:13.214588 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:13.214569    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-95438d51-5877d975f5-fj6t4"]
Jul 10 00:44:13.463762 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:13.463727    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:14.836496 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:14.836460    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4313d1f-2954-4935-a41c-9f9fab7ccf95" path="/var/lib/kubelet/pods/d4313d1f-2954-4935-a41c-9f9fab7ccf95/volumes"
Jul 10 00:44:17.639048 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:17.639016    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:17.693367 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:17.693335    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:17.851180 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:17.851146    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:17.947774 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:17.947740    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:18.945716 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:18.945681    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:19.017237 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.017202    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"]
Jul 10 00:44:19.017796 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.017776    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4313d1f-2954-4935-a41c-9f9fab7ccf95" containerName="maas-api"
Jul 10 00:44:19.017854 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.017801    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4313d1f-2954-4935-a41c-9f9fab7ccf95" containerName="maas-api"
Jul 10 00:44:19.017934 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.017922    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4313d1f-2954-4935-a41c-9f9fab7ccf95" containerName="maas-api"
Jul 10 00:44:19.021960 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.021926    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:19.022603 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.022579    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"
Jul 10 00:44:19.025229 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.025210    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-14264514-serving-cert\""
Jul 10 00:44:19.025487 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.025308    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vpn7x\""
Jul 10 00:44:19.030536 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.030518    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"]
Jul 10 00:44:19.097190 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.097158    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:19.142180 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.142142    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbqh9\" (UniqueName: \"kubernetes.io/projected/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-kube-api-access-mbqh9\") pod \"maas-api-e2e-ait-14264514-56fbb59dcc-52qnx\" (UID: \"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"
Jul 10 00:44:19.142180 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.142182    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-openshift-service-ca\") pod \"maas-api-e2e-ait-14264514-56fbb59dcc-52qnx\" (UID: \"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"
Jul 10 00:44:19.142409 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.142263    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-maas-api-tls\") pod \"maas-api-e2e-ait-14264514-56fbb59dcc-52qnx\" (UID: \"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"
Jul 10 00:44:19.243826 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.243730    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-maas-api-tls\") pod \"maas-api-e2e-ait-14264514-56fbb59dcc-52qnx\" (UID: \"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"
Jul 10 00:44:19.243983 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.243839    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbqh9\" (UniqueName: \"kubernetes.io/projected/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-kube-api-access-mbqh9\") pod \"maas-api-e2e-ait-14264514-56fbb59dcc-52qnx\" (UID: \"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"
Jul 10 00:44:19.243983 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.243867    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-openshift-service-ca\") pod \"maas-api-e2e-ait-14264514-56fbb59dcc-52qnx\" (UID: \"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"
Jul 10 00:44:19.243983 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:44:19.243881    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-14264514-serving-cert: secret "maas-api-e2e-ait-14264514-serving-cert" not found
Jul 10 00:44:19.243983 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:44:19.243960    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-maas-api-tls podName:5f71ef2d-78fc-46ca-a85a-2af36bc99c5e nodeName:}" failed. No retries permitted until 2026-07-10 00:44:19.743937285 +0000 UTC m=+1709.446722122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-maas-api-tls") pod "maas-api-e2e-ait-14264514-56fbb59dcc-52qnx" (UID: "5f71ef2d-78fc-46ca-a85a-2af36bc99c5e") : secret "maas-api-e2e-ait-14264514-serving-cert" not found
Jul 10 00:44:19.244590 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.244571    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-openshift-service-ca\") pod \"maas-api-e2e-ait-14264514-56fbb59dcc-52qnx\" (UID: \"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"
Jul 10 00:44:19.253564 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.253540    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbqh9\" (UniqueName: \"kubernetes.io/projected/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-kube-api-access-mbqh9\") pod \"maas-api-e2e-ait-14264514-56fbb59dcc-52qnx\" (UID: \"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"
Jul 10 00:44:19.303514 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.303471    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:19.484744 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.484708    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:19.590137 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.590048    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:19.747547 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.747503    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-maas-api-tls\") pod \"maas-api-e2e-ait-14264514-56fbb59dcc-52qnx\" (UID: \"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"
Jul 10 00:44:19.749972 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.749940    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-maas-api-tls\") pod \"maas-api-e2e-ait-14264514-56fbb59dcc-52qnx\" (UID: \"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"
Jul 10 00:44:19.936594 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:19.936498    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"
Jul 10 00:44:20.068226 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:20.068204    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"]
Jul 10 00:44:20.070068 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:44:20.070039    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f71ef2d_78fc_46ca_a85a_2af36bc99c5e.slice/crio-be82f159322acefd6f0e15ed630ee3219decb9de1839895695cabf3e2aeaa516 WatchSource:0}: Error finding container be82f159322acefd6f0e15ed630ee3219decb9de1839895695cabf3e2aeaa516: Status 404 returned error can't find the container with id be82f159322acefd6f0e15ed630ee3219decb9de1839895695cabf3e2aeaa516
Jul 10 00:44:20.228265 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:20.228237    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx" event={"ID":"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e","Type":"ContainerStarted","Data":"be82f159322acefd6f0e15ed630ee3219decb9de1839895695cabf3e2aeaa516"}
Jul 10 00:44:21.212734 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:21.212701    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:21.235487 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:21.235457    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx" event={"ID":"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e","Type":"ContainerStarted","Data":"fcc74dbfda5000a22e63ee49d41081ad5f3bbeea1d4220dfc9d2d8707849bed0"}
Jul 10 00:44:21.235651 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:21.235544    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"
Jul 10 00:44:21.253095 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:21.253042    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx" podStartSLOduration=2.81945371 podStartE2EDuration="3.253027056s" totalImagesPullingTime="433.573346ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:44:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:44:21.251811703 +0000 UTC m=+1710.954596581" watchObservedRunningTime="2026-07-10 00:44:21.253027056 +0000 UTC m=+1710.955811903"
Jul 10 00:44:24.125141 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:24.125109    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:24.580763 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:24.580728    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"]
Jul 10 00:44:24.581058 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:24.581026    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx" podUID="5f71ef2d-78fc-46ca-a85a-2af36bc99c5e" containerName="maas-api" containerID="cri-o://fcc74dbfda5000a22e63ee49d41081ad5f3bbeea1d4220dfc9d2d8707849bed0" gracePeriod=30
Jul 10 00:44:24.605156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:24.605130    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:24.826513 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:24.826490    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"
Jul 10 00:44:24.899363 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:24.899277    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-maas-api-tls\") pod \"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e\" (UID: \"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e\") "
Jul 10 00:44:24.899363 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:24.899337    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbqh9\" (UniqueName: \"kubernetes.io/projected/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-kube-api-access-mbqh9\") pod \"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e\" (UID: \"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e\") "
Jul 10 00:44:24.899585 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:24.899451    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-openshift-service-ca\") pod \"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e\" (UID: \"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e\") "
Jul 10 00:44:24.899910 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:24.899852    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5f71ef2d-78fc-46ca-a85a-2af36bc99c5e" (UID: "5f71ef2d-78fc-46ca-a85a-2af36bc99c5e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:44:24.901558 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:24.901534    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-kube-api-access-mbqh9" (OuterVolumeSpecName: "kube-api-access-mbqh9") pod "5f71ef2d-78fc-46ca-a85a-2af36bc99c5e" (UID: "5f71ef2d-78fc-46ca-a85a-2af36bc99c5e"). InnerVolumeSpecName "kube-api-access-mbqh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:44:24.901654 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:24.901554    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5f71ef2d-78fc-46ca-a85a-2af36bc99c5e" (UID: "5f71ef2d-78fc-46ca-a85a-2af36bc99c5e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:44:25.000866 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:25.000829    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-openshift-service-ca\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:44:25.000866 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:25.000854    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-maas-api-tls\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:44:25.000866 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:25.000863    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbqh9\" (UniqueName: \"kubernetes.io/projected/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e-kube-api-access-mbqh9\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:44:25.253292 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:25.253255    2559 generic.go:356] "Generic (PLEG): container finished" podID="5f71ef2d-78fc-46ca-a85a-2af36bc99c5e" containerID="fcc74dbfda5000a22e63ee49d41081ad5f3bbeea1d4220dfc9d2d8707849bed0" exitCode=0
Jul 10 00:44:25.253794 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:25.253297    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"
Jul 10 00:44:25.253794 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:25.253347    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx" event={"ID":"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e","Type":"ContainerDied","Data":"fcc74dbfda5000a22e63ee49d41081ad5f3bbeea1d4220dfc9d2d8707849bed0"}
Jul 10 00:44:25.253794 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:25.253412    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx" event={"ID":"5f71ef2d-78fc-46ca-a85a-2af36bc99c5e","Type":"ContainerDied","Data":"be82f159322acefd6f0e15ed630ee3219decb9de1839895695cabf3e2aeaa516"}
Jul 10 00:44:25.253794 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:25.253446    2559 scope.go:117] "RemoveContainer" containerID="fcc74dbfda5000a22e63ee49d41081ad5f3bbeea1d4220dfc9d2d8707849bed0"
Jul 10 00:44:25.264371 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:25.264353    2559 scope.go:117] "RemoveContainer" containerID="fcc74dbfda5000a22e63ee49d41081ad5f3bbeea1d4220dfc9d2d8707849bed0"
Jul 10 00:44:25.264643 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:44:25.264624    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcc74dbfda5000a22e63ee49d41081ad5f3bbeea1d4220dfc9d2d8707849bed0\": container with ID starting with fcc74dbfda5000a22e63ee49d41081ad5f3bbeea1d4220dfc9d2d8707849bed0 not found: ID does not exist" containerID="fcc74dbfda5000a22e63ee49d41081ad5f3bbeea1d4220dfc9d2d8707849bed0"
Jul 10 00:44:25.264710 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:25.264652    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcc74dbfda5000a22e63ee49d41081ad5f3bbeea1d4220dfc9d2d8707849bed0"} err="failed to get container status \"fcc74dbfda5000a22e63ee49d41081ad5f3bbeea1d4220dfc9d2d8707849bed0\": rpc error: code = NotFound desc = could not find container \"fcc74dbfda5000a22e63ee49d41081ad5f3bbeea1d4220dfc9d2d8707849bed0\": container with ID starting with fcc74dbfda5000a22e63ee49d41081ad5f3bbeea1d4220dfc9d2d8707849bed0 not found: ID does not exist"
Jul 10 00:44:25.274577 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:25.274557    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"]
Jul 10 00:44:25.278111 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:25.278089    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-14264514-56fbb59dcc-52qnx"]
Jul 10 00:44:26.836759 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:26.836728    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f71ef2d-78fc-46ca-a85a-2af36bc99c5e" path="/var/lib/kubelet/pods/5f71ef2d-78fc-46ca-a85a-2af36bc99c5e/volumes"
Jul 10 00:44:30.280120 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:30.280060    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:30.388819 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:30.388787    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:32.061732 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:32.061702    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:36.287727 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:36.287683    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:36.390784 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:36.390749    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:36.490237 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:36.490200    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:36.550445 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:36.550359    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:37.583992 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.583956    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:37.684369 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.684302    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:37.685398 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.685377    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"]
Jul 10 00:44:37.685759 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.685737    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f71ef2d-78fc-46ca-a85a-2af36bc99c5e" containerName="maas-api"
Jul 10 00:44:37.685759 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.685754    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f71ef2d-78fc-46ca-a85a-2af36bc99c5e" containerName="maas-api"
Jul 10 00:44:37.685855 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.685836    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f71ef2d-78fc-46ca-a85a-2af36bc99c5e" containerName="maas-api"
Jul 10 00:44:37.689972 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.689951    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"
Jul 10 00:44:37.692557 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.692535    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-055f1f02-serving-cert\""
Jul 10 00:44:37.692677 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.692538    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vpn7x\""
Jul 10 00:44:37.698257 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.698231    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"]
Jul 10 00:44:37.798241 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.798191    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:37.817945 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.817912    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da66a8c7-4f42-4163-8c3e-0688d047f4b1-maas-api-tls\") pod \"maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql\" (UID: \"da66a8c7-4f42-4163-8c3e-0688d047f4b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"
Jul 10 00:44:37.818097 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.817960    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jlm6\" (UniqueName: \"kubernetes.io/projected/da66a8c7-4f42-4163-8c3e-0688d047f4b1-kube-api-access-8jlm6\") pod \"maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql\" (UID: \"da66a8c7-4f42-4163-8c3e-0688d047f4b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"
Jul 10 00:44:37.818097 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.817987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da66a8c7-4f42-4163-8c3e-0688d047f4b1-openshift-service-ca\") pod \"maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql\" (UID: \"da66a8c7-4f42-4163-8c3e-0688d047f4b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"
Jul 10 00:44:37.919018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.918923    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da66a8c7-4f42-4163-8c3e-0688d047f4b1-maas-api-tls\") pod \"maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql\" (UID: \"da66a8c7-4f42-4163-8c3e-0688d047f4b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"
Jul 10 00:44:37.919018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.918980    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jlm6\" (UniqueName: \"kubernetes.io/projected/da66a8c7-4f42-4163-8c3e-0688d047f4b1-kube-api-access-8jlm6\") pod \"maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql\" (UID: \"da66a8c7-4f42-4163-8c3e-0688d047f4b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"
Jul 10 00:44:37.919018 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.919014    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da66a8c7-4f42-4163-8c3e-0688d047f4b1-openshift-service-ca\") pod \"maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql\" (UID: \"da66a8c7-4f42-4163-8c3e-0688d047f4b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"
Jul 10 00:44:37.919347 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:44:37.919100    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-serving-cert: secret "maas-api-e2e-derive-055f1f02-serving-cert" not found
Jul 10 00:44:37.919347 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:44:37.919195    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da66a8c7-4f42-4163-8c3e-0688d047f4b1-maas-api-tls podName:da66a8c7-4f42-4163-8c3e-0688d047f4b1 nodeName:}" failed. No retries permitted until 2026-07-10 00:44:38.419171558 +0000 UTC m=+1728.121956389 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/da66a8c7-4f42-4163-8c3e-0688d047f4b1-maas-api-tls") pod "maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql" (UID: "da66a8c7-4f42-4163-8c3e-0688d047f4b1") : secret "maas-api-e2e-derive-055f1f02-serving-cert" not found
Jul 10 00:44:37.919769 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.919747    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da66a8c7-4f42-4163-8c3e-0688d047f4b1-openshift-service-ca\") pod \"maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql\" (UID: \"da66a8c7-4f42-4163-8c3e-0688d047f4b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"
Jul 10 00:44:37.935458 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:37.935339    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jlm6\" (UniqueName: \"kubernetes.io/projected/da66a8c7-4f42-4163-8c3e-0688d047f4b1-kube-api-access-8jlm6\") pod \"maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql\" (UID: \"da66a8c7-4f42-4163-8c3e-0688d047f4b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"
Jul 10 00:44:38.131412 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:38.131376    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:38.245971 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:38.245937    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:38.347182 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:38.347143    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:38.424403 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:38.424368    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da66a8c7-4f42-4163-8c3e-0688d047f4b1-maas-api-tls\") pod \"maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql\" (UID: \"da66a8c7-4f42-4163-8c3e-0688d047f4b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"
Jul 10 00:44:38.426742 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:38.426720    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da66a8c7-4f42-4163-8c3e-0688d047f4b1-maas-api-tls\") pod \"maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql\" (UID: \"da66a8c7-4f42-4163-8c3e-0688d047f4b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"
Jul 10 00:44:38.602135 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:38.602048    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"
Jul 10 00:44:38.741109 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:38.741085    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"]
Jul 10 00:44:38.743860 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:44:38.743833    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda66a8c7_4f42_4163_8c3e_0688d047f4b1.slice/crio-38a193559d9b7b169fc446ed1ff450af2dec20d0242f48b7f79e5084c7e68928 WatchSource:0}: Error finding container 38a193559d9b7b169fc446ed1ff450af2dec20d0242f48b7f79e5084c7e68928: Status 404 returned error can't find the container with id 38a193559d9b7b169fc446ed1ff450af2dec20d0242f48b7f79e5084c7e68928
Jul 10 00:44:39.320525 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:39.320438    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql" event={"ID":"da66a8c7-4f42-4163-8c3e-0688d047f4b1","Type":"ContainerStarted","Data":"373e1799867e27c94767d615ea6b11ce89041ae2f22ce09b9785912453de3be9"}
Jul 10 00:44:39.320525 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:39.320490    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql" event={"ID":"da66a8c7-4f42-4163-8c3e-0688d047f4b1","Type":"ContainerStarted","Data":"38a193559d9b7b169fc446ed1ff450af2dec20d0242f48b7f79e5084c7e68928"}
Jul 10 00:44:39.320716 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:39.320569    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"
Jul 10 00:44:39.338440 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:39.338390    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql" podStartSLOduration=2.040270234 podStartE2EDuration="2.338373675s" totalImagesPullingTime="298.103441ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:44:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:44:39.337078445 +0000 UTC m=+1729.039863294" watchObservedRunningTime="2026-07-10 00:44:39.338373675 +0000 UTC m=+1729.041158524"
Jul 10 00:44:39.867524 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:39.867488    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:42.060069 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:42.060035    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:42.257902 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:42.257868    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:42.376668 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:42.376582    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:42.453855 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:42.453824    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"]
Jul 10 00:44:42.454239 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:42.454206    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql" podUID="da66a8c7-4f42-4163-8c3e-0688d047f4b1" containerName="maas-api" containerID="cri-o://373e1799867e27c94767d615ea6b11ce89041ae2f22ce09b9785912453de3be9" gracePeriod=30
Jul 10 00:44:42.476933 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:42.476907    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:42.708173 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:42.708149    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"
Jul 10 00:44:42.759503 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:42.759475    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da66a8c7-4f42-4163-8c3e-0688d047f4b1-openshift-service-ca\") pod \"da66a8c7-4f42-4163-8c3e-0688d047f4b1\" (UID: \"da66a8c7-4f42-4163-8c3e-0688d047f4b1\") "
Jul 10 00:44:42.759503 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:42.759508    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da66a8c7-4f42-4163-8c3e-0688d047f4b1-maas-api-tls\") pod \"da66a8c7-4f42-4163-8c3e-0688d047f4b1\" (UID: \"da66a8c7-4f42-4163-8c3e-0688d047f4b1\") "
Jul 10 00:44:42.759707 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:42.759535    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jlm6\" (UniqueName: \"kubernetes.io/projected/da66a8c7-4f42-4163-8c3e-0688d047f4b1-kube-api-access-8jlm6\") pod \"da66a8c7-4f42-4163-8c3e-0688d047f4b1\" (UID: \"da66a8c7-4f42-4163-8c3e-0688d047f4b1\") "
Jul 10 00:44:42.759855 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:42.759835    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da66a8c7-4f42-4163-8c3e-0688d047f4b1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "da66a8c7-4f42-4163-8c3e-0688d047f4b1" (UID: "da66a8c7-4f42-4163-8c3e-0688d047f4b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:44:42.761643 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:42.761618    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da66a8c7-4f42-4163-8c3e-0688d047f4b1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "da66a8c7-4f42-4163-8c3e-0688d047f4b1" (UID: "da66a8c7-4f42-4163-8c3e-0688d047f4b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:44:42.761744 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:42.761668    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da66a8c7-4f42-4163-8c3e-0688d047f4b1-kube-api-access-8jlm6" (OuterVolumeSpecName: "kube-api-access-8jlm6") pod "da66a8c7-4f42-4163-8c3e-0688d047f4b1" (UID: "da66a8c7-4f42-4163-8c3e-0688d047f4b1"). InnerVolumeSpecName "kube-api-access-8jlm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:44:42.860473 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:42.860450    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da66a8c7-4f42-4163-8c3e-0688d047f4b1-openshift-service-ca\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:44:42.860473 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:42.860472    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da66a8c7-4f42-4163-8c3e-0688d047f4b1-maas-api-tls\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:44:42.860693 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:42.860482    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jlm6\" (UniqueName: \"kubernetes.io/projected/da66a8c7-4f42-4163-8c3e-0688d047f4b1-kube-api-access-8jlm6\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:44:43.339983 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:43.339944    2559 generic.go:356] "Generic (PLEG): container finished" podID="da66a8c7-4f42-4163-8c3e-0688d047f4b1" containerID="373e1799867e27c94767d615ea6b11ce89041ae2f22ce09b9785912453de3be9" exitCode=0
Jul 10 00:44:43.340407 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:43.339993    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"
Jul 10 00:44:43.340407 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:43.340026    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql" event={"ID":"da66a8c7-4f42-4163-8c3e-0688d047f4b1","Type":"ContainerDied","Data":"373e1799867e27c94767d615ea6b11ce89041ae2f22ce09b9785912453de3be9"}
Jul 10 00:44:43.340407 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:43.340078    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql" event={"ID":"da66a8c7-4f42-4163-8c3e-0688d047f4b1","Type":"ContainerDied","Data":"38a193559d9b7b169fc446ed1ff450af2dec20d0242f48b7f79e5084c7e68928"}
Jul 10 00:44:43.340407 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:43.340093    2559 scope.go:117] "RemoveContainer" containerID="373e1799867e27c94767d615ea6b11ce89041ae2f22ce09b9785912453de3be9"
Jul 10 00:44:43.348449 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:43.348431    2559 scope.go:117] "RemoveContainer" containerID="373e1799867e27c94767d615ea6b11ce89041ae2f22ce09b9785912453de3be9"
Jul 10 00:44:43.348726 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:44:43.348705    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"373e1799867e27c94767d615ea6b11ce89041ae2f22ce09b9785912453de3be9\": container with ID starting with 373e1799867e27c94767d615ea6b11ce89041ae2f22ce09b9785912453de3be9 not found: ID does not exist" containerID="373e1799867e27c94767d615ea6b11ce89041ae2f22ce09b9785912453de3be9"
Jul 10 00:44:43.348838 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:43.348737    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"373e1799867e27c94767d615ea6b11ce89041ae2f22ce09b9785912453de3be9"} err="failed to get container status \"373e1799867e27c94767d615ea6b11ce89041ae2f22ce09b9785912453de3be9\": rpc error: code = NotFound desc = could not find container \"373e1799867e27c94767d615ea6b11ce89041ae2f22ce09b9785912453de3be9\": container with ID starting with 373e1799867e27c94767d615ea6b11ce89041ae2f22ce09b9785912453de3be9 not found: ID does not exist"
Jul 10 00:44:43.355803 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:43.355779    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"]
Jul 10 00:44:43.359260 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:43.359240    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-055f1f02-6d9c6f8844-cfxql"]
Jul 10 00:44:43.704636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:43.704601    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:44.836744 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:44.836716    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da66a8c7-4f42-4163-8c3e-0688d047f4b1" path="/var/lib/kubelet/pods/da66a8c7-4f42-4163-8c3e-0688d047f4b1/volumes"
Jul 10 00:44:48.105368 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:48.105335    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:48.206166 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:48.206134    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:48.587571 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:48.587537    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:48.696038 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:48.696000    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:48.800913 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:48.800863    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:50.232788 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:50.232745    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:55.813970 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:55.813936    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:55.871638 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:55.871598    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:55.949501 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:55.949464    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:56.043995 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:56.043954    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:56.100811 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:56.100715    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:44:56.588963 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:44:56.588927    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:45:00.134395 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:00.134362    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727405-vmhnn"]
Jul 10 00:45:00.134750 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:00.134707    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da66a8c7-4f42-4163-8c3e-0688d047f4b1" containerName="maas-api"
Jul 10 00:45:00.134750 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:00.134718    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="da66a8c7-4f42-4163-8c3e-0688d047f4b1" containerName="maas-api"
Jul 10 00:45:00.134821 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:00.134777    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="da66a8c7-4f42-4163-8c3e-0688d047f4b1" containerName="maas-api"
Jul 10 00:45:00.137623 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:00.137606    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-vmhnn"
Jul 10 00:45:00.140145 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:00.140125    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gfl2c\""
Jul 10 00:45:00.143921 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:00.143901    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727405-vmhnn"]
Jul 10 00:45:00.148721 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:00.148698    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-vmhnn"
Jul 10 00:45:00.282360 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:00.282338    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727405-vmhnn"]
Jul 10 00:45:00.285220 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:45:00.285191    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf042dc_8295_4ced_8866_b58585130b1a.slice/crio-fab64ac3218e9edb48d59306d29b88074afefda2a3b841fce6d8bcc84d4126f3 WatchSource:0}: Error finding container fab64ac3218e9edb48d59306d29b88074afefda2a3b841fce6d8bcc84d4126f3: Status 404 returned error can't find the container with id fab64ac3218e9edb48d59306d29b88074afefda2a3b841fce6d8bcc84d4126f3
Jul 10 00:45:00.423766 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:00.423686    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-vmhnn" event={"ID":"0cf042dc-8295-4ced-8866-b58585130b1a","Type":"ContainerStarted","Data":"fab64ac3218e9edb48d59306d29b88074afefda2a3b841fce6d8bcc84d4126f3"}
Jul 10 00:45:01.429841 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:01.429801    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-vmhnn" event={"ID":"0cf042dc-8295-4ced-8866-b58585130b1a","Type":"ContainerStarted","Data":"e776a6540bdfd38b27c25398b3af85bc37273aa222c47529f42d97c71414cff5"}
Jul 10 00:45:01.445429 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:01.445373    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-vmhnn" podStartSLOduration=1.445357368 podStartE2EDuration="1.445357368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:45:01.443686958 +0000 UTC m=+1751.146471806" watchObservedRunningTime="2026-07-10 00:45:01.445357368 +0000 UTC m=+1751.148142216"
Jul 10 00:45:22.676405 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:22.676362    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:45:28.347831 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:28.347788    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:45:28.410852 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:28.410798    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:45:28.496790 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:28.496761    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:45:28.596169 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:28.596132    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:45:44.666600 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:44.666505    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:45:50.200285 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:50.200217    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:45:50.249134 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:50.249098    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:45:50.380823 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:50.380773    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:45:50.474763 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:50.474664    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:45:50.838286 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:50.838205    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:45:50.842006 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:45:50.841986    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:46:00.010647 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:00.010608    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727405-vmhnn"]
Jul 10 00:46:00.011059 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:00.010824    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-vmhnn" podUID="0cf042dc-8295-4ced-8866-b58585130b1a" containerName="cleanup" containerID="cri-o://e776a6540bdfd38b27c25398b3af85bc37273aa222c47529f42d97c71414cff5" gracePeriod=30
Jul 10 00:46:01.063675 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:01.063645    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:01.926941 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:01.926908    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:07.570747 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:07.570704    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:07.629772 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:07.629727    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:07.707890 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:07.707858    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:07.805920 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:07.805875    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:07.906665 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:07.906565    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:08.793167 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:08.793133    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:08.892941 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:08.892903    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:08.993117 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:08.993080    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:10.720954 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:10.720916    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:16.437197 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:16.437163    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:16.495963 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:16.495934    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:17.118445 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:17.118407    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:22.741944 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:22.741907    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:22.800116 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:22.800081    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:22.983907 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:22.983876    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:23.294850 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:23.294815    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:23.509633 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:23.509597    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:26.601092 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.601051    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:26.686331 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.686280    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:26.688402 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.688364    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"]
Jul 10 00:46:26.691976 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.691953    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"
Jul 10 00:46:26.694673 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.694654    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-fa85739d-serving-cert\""
Jul 10 00:46:26.694802 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.694774    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vpn7x\""
Jul 10 00:46:26.699200 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.699178    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"]
Jul 10 00:46:26.747103 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.747064    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:26.772274 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.772244    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8xdc\" (UniqueName: \"kubernetes.io/projected/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-kube-api-access-s8xdc\") pod \"maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv\" (UID: \"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"
Jul 10 00:46:26.772401 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.772280    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-openshift-service-ca\") pod \"maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv\" (UID: \"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"
Jul 10 00:46:26.772452 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.772414    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-maas-api-tls\") pod \"maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv\" (UID: \"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"
Jul 10 00:46:26.847078 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.847043    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:26.873369 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.873286    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-maas-api-tls\") pod \"maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv\" (UID: \"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"
Jul 10 00:46:26.873464 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.873375    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8xdc\" (UniqueName: \"kubernetes.io/projected/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-kube-api-access-s8xdc\") pod \"maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv\" (UID: \"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"
Jul 10 00:46:26.873464 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.873404    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-openshift-service-ca\") pod \"maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv\" (UID: \"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"
Jul 10 00:46:26.874163 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.874128    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-openshift-service-ca\") pod \"maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv\" (UID: \"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"
Jul 10 00:46:26.876362 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.876341    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-maas-api-tls\") pod \"maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv\" (UID: \"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"
Jul 10 00:46:26.880865 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:26.880845    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8xdc\" (UniqueName: \"kubernetes.io/projected/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-kube-api-access-s8xdc\") pod \"maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv\" (UID: \"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"
Jul 10 00:46:27.004514 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:27.004472    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"
Jul 10 00:46:27.141740 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:27.141646    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"]
Jul 10 00:46:27.144549 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:46:27.144520    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9548fe78_6fcf_49ad_b2e2_58d1e1518ab1.slice/crio-8e4c77caf325fc25608dfe76c1d97f7a6f4ccfc6f193cd2040c6ad4207d76ea0 WatchSource:0}: Error finding container 8e4c77caf325fc25608dfe76c1d97f7a6f4ccfc6f193cd2040c6ad4207d76ea0: Status 404 returned error can't find the container with id 8e4c77caf325fc25608dfe76c1d97f7a6f4ccfc6f193cd2040c6ad4207d76ea0
Jul 10 00:46:27.157743 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:27.157717    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:27.257995 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:27.257955    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:27.823693 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:27.823604    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv" event={"ID":"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1","Type":"ContainerStarted","Data":"106ce4b51a2ed4c26ce8f8facc77ec3ffe1ebff68477c8807cfb96ecd1d2a617"}
Jul 10 00:46:27.823693 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:27.823663    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv" event={"ID":"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1","Type":"ContainerStarted","Data":"8e4c77caf325fc25608dfe76c1d97f7a6f4ccfc6f193cd2040c6ad4207d76ea0"}
Jul 10 00:46:27.824084 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:27.823738    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"
Jul 10 00:46:27.840267 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:27.840215    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv" podStartSLOduration=1.439507113 podStartE2EDuration="1.840195251s" totalImagesPullingTime="400.688138ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:46:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:46:27.839036291 +0000 UTC m=+1837.541821139" watchObservedRunningTime="2026-07-10 00:46:27.840195251 +0000 UTC m=+1837.542980099"
Jul 10 00:46:30.183702 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:30.183682    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-vmhnn"
Jul 10 00:46:30.837937 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:30.837898    2559 generic.go:356] "Generic (PLEG): container finished" podID="0cf042dc-8295-4ced-8866-b58585130b1a" containerID="e776a6540bdfd38b27c25398b3af85bc37273aa222c47529f42d97c71414cff5" exitCode=137
Jul 10 00:46:30.838108 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:30.837949    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-vmhnn" event={"ID":"0cf042dc-8295-4ced-8866-b58585130b1a","Type":"ContainerDied","Data":"e776a6540bdfd38b27c25398b3af85bc37273aa222c47529f42d97c71414cff5"}
Jul 10 00:46:30.838108 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:30.837987    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-vmhnn" event={"ID":"0cf042dc-8295-4ced-8866-b58585130b1a","Type":"ContainerDied","Data":"fab64ac3218e9edb48d59306d29b88074afefda2a3b841fce6d8bcc84d4126f3"}
Jul 10 00:46:30.838108 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:30.838009    2559 scope.go:117] "RemoveContainer" containerID="e776a6540bdfd38b27c25398b3af85bc37273aa222c47529f42d97c71414cff5"
Jul 10 00:46:30.838108 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:30.837955    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727405-vmhnn"
Jul 10 00:46:30.846347 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:30.846329    2559 scope.go:117] "RemoveContainer" containerID="e776a6540bdfd38b27c25398b3af85bc37273aa222c47529f42d97c71414cff5"
Jul 10 00:46:30.846576 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:46:30.846557    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e776a6540bdfd38b27c25398b3af85bc37273aa222c47529f42d97c71414cff5\": container with ID starting with e776a6540bdfd38b27c25398b3af85bc37273aa222c47529f42d97c71414cff5 not found: ID does not exist" containerID="e776a6540bdfd38b27c25398b3af85bc37273aa222c47529f42d97c71414cff5"
Jul 10 00:46:30.846636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:30.846583    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e776a6540bdfd38b27c25398b3af85bc37273aa222c47529f42d97c71414cff5"} err="failed to get container status \"e776a6540bdfd38b27c25398b3af85bc37273aa222c47529f42d97c71414cff5\": rpc error: code = NotFound desc = could not find container \"e776a6540bdfd38b27c25398b3af85bc37273aa222c47529f42d97c71414cff5\": container with ID starting with e776a6540bdfd38b27c25398b3af85bc37273aa222c47529f42d97c71414cff5 not found: ID does not exist"
Jul 10 00:46:30.859581 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:30.859551    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727405-vmhnn"]
Jul 10 00:46:30.862909 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:30.862888    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727405-vmhnn"]
Jul 10 00:46:31.137195 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:31.137107    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:31.805940 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:31.805906    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:31.904061 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:31.904021    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:32.254038 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:32.254004    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:32.836158 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:32.836127    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cf042dc-8295-4ced-8866-b58585130b1a" path="/var/lib/kubelet/pods/0cf042dc-8295-4ced-8866-b58585130b1a/volumes"
Jul 10 00:46:33.833366 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:33.833337    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"
Jul 10 00:46:38.015551 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:38.015517    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:38.070855 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:38.070819    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:38.398780 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:38.398685    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:38.703812 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:38.703751    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"]
Jul 10 00:46:38.704448 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:38.704012    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv" podUID="9548fe78-6fcf-49ad-b2e2-58d1e1518ab1" containerName="maas-api" containerID="cri-o://106ce4b51a2ed4c26ce8f8facc77ec3ffe1ebff68477c8807cfb96ecd1d2a617" gracePeriod=30
Jul 10 00:46:38.728046 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:38.728019    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:38.827679 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:38.827613    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:38.828510 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:38.828474    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv" podUID="9548fe78-6fcf-49ad-b2e2-58d1e1518ab1" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.39:8443/health\": dial tcp 10.134.0.39:8443: connect: connection refused"
Jul 10 00:46:38.880832 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:38.880785    2559 generic.go:356] "Generic (PLEG): container finished" podID="9548fe78-6fcf-49ad-b2e2-58d1e1518ab1" containerID="106ce4b51a2ed4c26ce8f8facc77ec3ffe1ebff68477c8807cfb96ecd1d2a617" exitCode=0
Jul 10 00:46:38.880973 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:38.880859    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv" event={"ID":"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1","Type":"ContainerDied","Data":"106ce4b51a2ed4c26ce8f8facc77ec3ffe1ebff68477c8807cfb96ecd1d2a617"}
Jul 10 00:46:38.964224 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:38.964165    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"
Jul 10 00:46:39.083445 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:39.083397    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-openshift-service-ca\") pod \"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1\" (UID: \"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1\") "
Jul 10 00:46:39.083907 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:39.083508    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8xdc\" (UniqueName: \"kubernetes.io/projected/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-kube-api-access-s8xdc\") pod \"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1\" (UID: \"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1\") "
Jul 10 00:46:39.083907 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:39.083560    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-maas-api-tls\") pod \"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1\" (UID: \"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1\") "
Jul 10 00:46:39.083907 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:39.083827    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9548fe78-6fcf-49ad-b2e2-58d1e1518ab1" (UID: "9548fe78-6fcf-49ad-b2e2-58d1e1518ab1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:46:39.085624 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:39.085598    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9548fe78-6fcf-49ad-b2e2-58d1e1518ab1" (UID: "9548fe78-6fcf-49ad-b2e2-58d1e1518ab1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:46:39.085696 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:39.085667    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-kube-api-access-s8xdc" (OuterVolumeSpecName: "kube-api-access-s8xdc") pod "9548fe78-6fcf-49ad-b2e2-58d1e1518ab1" (UID: "9548fe78-6fcf-49ad-b2e2-58d1e1518ab1"). InnerVolumeSpecName "kube-api-access-s8xdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:46:39.184360 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:39.184302    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8xdc\" (UniqueName: \"kubernetes.io/projected/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-kube-api-access-s8xdc\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:46:39.184360 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:39.184355    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-maas-api-tls\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:46:39.184360 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:39.184365    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1-openshift-service-ca\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:46:39.886680 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:39.886654    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"
Jul 10 00:46:39.886824 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:39.886654    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv" event={"ID":"9548fe78-6fcf-49ad-b2e2-58d1e1518ab1","Type":"ContainerDied","Data":"8e4c77caf325fc25608dfe76c1d97f7a6f4ccfc6f193cd2040c6ad4207d76ea0"}
Jul 10 00:46:39.886824 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:39.886795    2559 scope.go:117] "RemoveContainer" containerID="106ce4b51a2ed4c26ce8f8facc77ec3ffe1ebff68477c8807cfb96ecd1d2a617"
Jul 10 00:46:39.907059 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:39.907030    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"]
Jul 10 00:46:39.910572 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:39.910551    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fa85739d-6bcc9cd98b-lg9jv"]
Jul 10 00:46:40.836785 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:40.836752    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9548fe78-6fcf-49ad-b2e2-58d1e1518ab1" path="/var/lib/kubelet/pods/9548fe78-6fcf-49ad-b2e2-58d1e1518ab1/volumes"
Jul 10 00:46:44.529438 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:44.529398    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:44.631485 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:44.631445    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:44.891162 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:44.891065    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:45.047987 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:45.047947    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:45.174982 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:45.174847    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:45.267235 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:45.267197    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:45.405306 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:45.405270    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:51.089546 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:51.089504    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:51.194292 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:51.194264    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:51.290914 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:51.290878    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:51.428363 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:51.428292    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:57.199107 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:57.199062    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:46:57.254422 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:46:57.254386    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:04.033178 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:04.033141    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:04.333462 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:04.333377    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:04.739619 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:04.739579    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:07.813890 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.813858    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:07.874944 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.874917    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"]
Jul 10 00:47:07.875283 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.875271    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9548fe78-6fcf-49ad-b2e2-58d1e1518ab1" containerName="maas-api"
Jul 10 00:47:07.875343 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.875285    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9548fe78-6fcf-49ad-b2e2-58d1e1518ab1" containerName="maas-api"
Jul 10 00:47:07.875343 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.875296    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cf042dc-8295-4ced-8866-b58585130b1a" containerName="cleanup"
Jul 10 00:47:07.875343 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.875302    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cf042dc-8295-4ced-8866-b58585130b1a" containerName="cleanup"
Jul 10 00:47:07.875445 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.875369    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9548fe78-6fcf-49ad-b2e2-58d1e1518ab1" containerName="maas-api"
Jul 10 00:47:07.875445 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.875381    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cf042dc-8295-4ced-8866-b58585130b1a" containerName="cleanup"
Jul 10 00:47:07.878402 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.878370    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"
Jul 10 00:47:07.880421 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.880392    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:07.881258 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.881225    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9a8ef25d-serving-cert\""
Jul 10 00:47:07.881508 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.881492    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vpn7x\""
Jul 10 00:47:07.888918 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.888898    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"]
Jul 10 00:47:07.954961 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.954924    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd5640be-f40d-4643-98f3-8d90555d3290-openshift-service-ca\") pod \"maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84\" (UID: \"fd5640be-f40d-4643-98f3-8d90555d3290\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"
Jul 10 00:47:07.955120 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.954967    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd5640be-f40d-4643-98f3-8d90555d3290-maas-api-tls\") pod \"maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84\" (UID: \"fd5640be-f40d-4643-98f3-8d90555d3290\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"
Jul 10 00:47:07.955120 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.955005    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m62np\" (UniqueName: \"kubernetes.io/projected/fd5640be-f40d-4643-98f3-8d90555d3290-kube-api-access-m62np\") pod \"maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84\" (UID: \"fd5640be-f40d-4643-98f3-8d90555d3290\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"
Jul 10 00:47:07.959754 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:07.959719    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:08.056064 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:08.056020    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd5640be-f40d-4643-98f3-8d90555d3290-openshift-service-ca\") pod \"maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84\" (UID: \"fd5640be-f40d-4643-98f3-8d90555d3290\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"
Jul 10 00:47:08.056064 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:08.056067    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd5640be-f40d-4643-98f3-8d90555d3290-maas-api-tls\") pod \"maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84\" (UID: \"fd5640be-f40d-4643-98f3-8d90555d3290\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"
Jul 10 00:47:08.056345 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:08.056099    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m62np\" (UniqueName: \"kubernetes.io/projected/fd5640be-f40d-4643-98f3-8d90555d3290-kube-api-access-m62np\") pod \"maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84\" (UID: \"fd5640be-f40d-4643-98f3-8d90555d3290\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"
Jul 10 00:47:08.056345 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:47:08.056254    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-serving-cert: secret "maas-api-e2e-mt-9a8ef25d-serving-cert" not found
Jul 10 00:47:08.056464 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:47:08.056348    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd5640be-f40d-4643-98f3-8d90555d3290-maas-api-tls podName:fd5640be-f40d-4643-98f3-8d90555d3290 nodeName:}" failed. No retries permitted until 2026-07-10 00:47:08.556325101 +0000 UTC m=+1878.259109942 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fd5640be-f40d-4643-98f3-8d90555d3290-maas-api-tls") pod "maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84" (UID: "fd5640be-f40d-4643-98f3-8d90555d3290") : secret "maas-api-e2e-mt-9a8ef25d-serving-cert" not found
Jul 10 00:47:08.056724 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:08.056704    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd5640be-f40d-4643-98f3-8d90555d3290-openshift-service-ca\") pod \"maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84\" (UID: \"fd5640be-f40d-4643-98f3-8d90555d3290\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"
Jul 10 00:47:08.065222 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:08.065159    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m62np\" (UniqueName: \"kubernetes.io/projected/fd5640be-f40d-4643-98f3-8d90555d3290-kube-api-access-m62np\") pod \"maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84\" (UID: \"fd5640be-f40d-4643-98f3-8d90555d3290\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"
Jul 10 00:47:08.221173 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:08.221133    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:08.335281 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:08.335185    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:08.438772 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:08.438733    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:08.560432 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:08.560397    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd5640be-f40d-4643-98f3-8d90555d3290-maas-api-tls\") pod \"maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84\" (UID: \"fd5640be-f40d-4643-98f3-8d90555d3290\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"
Jul 10 00:47:08.562994 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:08.562961    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd5640be-f40d-4643-98f3-8d90555d3290-maas-api-tls\") pod \"maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84\" (UID: \"fd5640be-f40d-4643-98f3-8d90555d3290\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"
Jul 10 00:47:08.792996 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:08.792952    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"
Jul 10 00:47:08.929941 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:08.929918    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"]
Jul 10 00:47:08.932708 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:47:08.932682    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd5640be_f40d_4643_98f3_8d90555d3290.slice/crio-20fd78717b7b93100f52b7cc5347458fec0bd0be3125603b505880bce284a6b4 WatchSource:0}: Error finding container 20fd78717b7b93100f52b7cc5347458fec0bd0be3125603b505880bce284a6b4: Status 404 returned error can't find the container with id 20fd78717b7b93100f52b7cc5347458fec0bd0be3125603b505880bce284a6b4
Jul 10 00:47:09.021210 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:09.021174    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84" event={"ID":"fd5640be-f40d-4643-98f3-8d90555d3290","Type":"ContainerStarted","Data":"20fd78717b7b93100f52b7cc5347458fec0bd0be3125603b505880bce284a6b4"}
Jul 10 00:47:10.028353 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:10.028296    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84" event={"ID":"fd5640be-f40d-4643-98f3-8d90555d3290","Type":"ContainerStarted","Data":"32016b3edc83d51d89780e943febee28a136355e5ae72fe4228d78af8185bf87"}
Jul 10 00:47:10.028741 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:10.028451    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"
Jul 10 00:47:10.045918 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:10.045869    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84" podStartSLOduration=2.603989325 podStartE2EDuration="3.045853571s" totalImagesPullingTime="441.864246ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:47:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:47:10.04391261 +0000 UTC m=+1879.746697459" watchObservedRunningTime="2026-07-10 00:47:10.045853571 +0000 UTC m=+1879.748638419"
Jul 10 00:47:12.317893 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:12.317809    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:16.038286 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:16.038252    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"
Jul 10 00:47:21.176178 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:21.176139    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:21.277114 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:21.277076    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:21.400823 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:21.400793    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:27.906251 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:27.906215    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:28.007073 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:28.007035    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:29.015079 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:29.015038    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:29.115610 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:29.115575    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:45.746846 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:45.746810    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:46.027201 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:46.027087    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"]
Jul 10 00:47:46.027422 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:46.027387    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84" podUID="fd5640be-f40d-4643-98f3-8d90555d3290" containerName="maas-api" containerID="cri-o://32016b3edc83d51d89780e943febee28a136355e5ae72fe4228d78af8185bf87" gracePeriod=30
Jul 10 00:47:46.033254 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:46.033204    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84" podUID="fd5640be-f40d-4643-98f3-8d90555d3290" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.40:8443/health\": dial tcp 10.134.0.40:8443: connect: connection refused"
Jul 10 00:47:46.054962 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:46.054932    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:46.199352 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:46.199297    2559 generic.go:356] "Generic (PLEG): container finished" podID="fd5640be-f40d-4643-98f3-8d90555d3290" containerID="32016b3edc83d51d89780e943febee28a136355e5ae72fe4228d78af8185bf87" exitCode=0
Jul 10 00:47:46.199533 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:46.199422    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84" event={"ID":"fd5640be-f40d-4643-98f3-8d90555d3290","Type":"ContainerDied","Data":"32016b3edc83d51d89780e943febee28a136355e5ae72fe4228d78af8185bf87"}
Jul 10 00:47:46.269915 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:46.269890    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"
Jul 10 00:47:46.308548 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:46.308455    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd5640be-f40d-4643-98f3-8d90555d3290-maas-api-tls\") pod \"fd5640be-f40d-4643-98f3-8d90555d3290\" (UID: \"fd5640be-f40d-4643-98f3-8d90555d3290\") "
Jul 10 00:47:46.308548 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:46.308529    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd5640be-f40d-4643-98f3-8d90555d3290-openshift-service-ca\") pod \"fd5640be-f40d-4643-98f3-8d90555d3290\" (UID: \"fd5640be-f40d-4643-98f3-8d90555d3290\") "
Jul 10 00:47:46.308780 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:46.308598    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m62np\" (UniqueName: \"kubernetes.io/projected/fd5640be-f40d-4643-98f3-8d90555d3290-kube-api-access-m62np\") pod \"fd5640be-f40d-4643-98f3-8d90555d3290\" (UID: \"fd5640be-f40d-4643-98f3-8d90555d3290\") "
Jul 10 00:47:46.308945 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:46.308914    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd5640be-f40d-4643-98f3-8d90555d3290-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fd5640be-f40d-4643-98f3-8d90555d3290" (UID: "fd5640be-f40d-4643-98f3-8d90555d3290"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:47:46.311224 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:46.311201    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd5640be-f40d-4643-98f3-8d90555d3290-kube-api-access-m62np" (OuterVolumeSpecName: "kube-api-access-m62np") pod "fd5640be-f40d-4643-98f3-8d90555d3290" (UID: "fd5640be-f40d-4643-98f3-8d90555d3290"). InnerVolumeSpecName "kube-api-access-m62np". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:47:46.311224 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:46.311208    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd5640be-f40d-4643-98f3-8d90555d3290-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fd5640be-f40d-4643-98f3-8d90555d3290" (UID: "fd5640be-f40d-4643-98f3-8d90555d3290"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:47:46.409461 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:46.409425    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd5640be-f40d-4643-98f3-8d90555d3290-openshift-service-ca\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:47:46.409461 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:46.409452    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m62np\" (UniqueName: \"kubernetes.io/projected/fd5640be-f40d-4643-98f3-8d90555d3290-kube-api-access-m62np\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:47:46.409461 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:46.409464    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd5640be-f40d-4643-98f3-8d90555d3290-maas-api-tls\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:47:47.205224 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:47.205185    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84" event={"ID":"fd5640be-f40d-4643-98f3-8d90555d3290","Type":"ContainerDied","Data":"20fd78717b7b93100f52b7cc5347458fec0bd0be3125603b505880bce284a6b4"}
Jul 10 00:47:47.205224 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:47.205220    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"
Jul 10 00:47:47.205740 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:47.205262    2559 scope.go:117] "RemoveContainer" containerID="32016b3edc83d51d89780e943febee28a136355e5ae72fe4228d78af8185bf87"
Jul 10 00:47:47.222019 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:47.221901    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"]
Jul 10 00:47:47.223857 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:47.223832    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9a8ef25d-57545b96bf-mbs84"]
Jul 10 00:47:48.835824 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:48.835792    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd5640be-f40d-4643-98f3-8d90555d3290" path="/var/lib/kubelet/pods/fd5640be-f40d-4643-98f3-8d90555d3290/volumes"
Jul 10 00:47:51.861453 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:51.861418    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:47:56.936689 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:47:56.936646    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:48:33.118573 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:48:33.118535    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:48:33.887015 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:48:33.886980    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:48:39.468459 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:48:39.468406    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:48:53.486615 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:48:53.486536    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:48:53.587259 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:48:53.587218    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:48:53.773543 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:48:53.773451    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:48:54.035469 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:48:54.035377    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:48:55.134799 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:48:55.134754    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:48:55.232827 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:48:55.232787    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:48:55.334046 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:48:55.334013    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:48:56.105850 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:48:56.105800    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:48:56.205095 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:48:56.205054    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:48:56.304909 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:48:56.304871    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:48:57.744185 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:48:57.744145    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:03.306494 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:03.306452    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:03.406546 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:03.406514    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:03.998291 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:03.998248    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:09.801296 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:09.801256    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:09.856634 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:09.856601    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:09.938524 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:09.938493    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:10.038519 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:10.038470    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:20.548565 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:20.548527    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:20.910954 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:20.910863    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:31.523751 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:31.523715    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:41.912019 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:41.911976    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:47.639086 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:47.639048    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:47.739558 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:47.739526    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:47.977569 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:47.977528    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:48.316149 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:48.316071    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:48.614951 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:48.614858    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:51.667152 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.667109    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:51.742772 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.742739    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"]
Jul 10 00:49:51.743372 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.743351    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd5640be-f40d-4643-98f3-8d90555d3290" containerName="maas-api"
Jul 10 00:49:51.743372 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.743373    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd5640be-f40d-4643-98f3-8d90555d3290" containerName="maas-api"
Jul 10 00:49:51.743554 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.743481    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd5640be-f40d-4643-98f3-8d90555d3290" containerName="maas-api"
Jul 10 00:49:51.744359 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.744328    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:51.746894 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.746875    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"
Jul 10 00:49:51.749517 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.749496    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-b8817a-serving-cert\""
Jul 10 00:49:51.749630 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.749515    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vpn7x\""
Jul 10 00:49:51.757100 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.757076    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"]
Jul 10 00:49:51.805297 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.805257    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:51.828050 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.828021    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c3a56af-364f-4e10-b1fd-999c985d527f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b8817a-779659c584-jggpp\" (UID: \"5c3a56af-364f-4e10-b1fd-999c985d527f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"
Jul 10 00:49:51.828191 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.828056    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t9k8\" (UniqueName: \"kubernetes.io/projected/5c3a56af-364f-4e10-b1fd-999c985d527f-kube-api-access-5t9k8\") pod \"maas-api-e2e-inf-a-b8817a-779659c584-jggpp\" (UID: \"5c3a56af-364f-4e10-b1fd-999c985d527f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"
Jul 10 00:49:51.828191 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.828079    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c3a56af-364f-4e10-b1fd-999c985d527f-maas-api-tls\") pod \"maas-api-e2e-inf-a-b8817a-779659c584-jggpp\" (UID: \"5c3a56af-364f-4e10-b1fd-999c985d527f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"
Jul 10 00:49:51.929639 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.929538    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c3a56af-364f-4e10-b1fd-999c985d527f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b8817a-779659c584-jggpp\" (UID: \"5c3a56af-364f-4e10-b1fd-999c985d527f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"
Jul 10 00:49:51.929639 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.929585    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t9k8\" (UniqueName: \"kubernetes.io/projected/5c3a56af-364f-4e10-b1fd-999c985d527f-kube-api-access-5t9k8\") pod \"maas-api-e2e-inf-a-b8817a-779659c584-jggpp\" (UID: \"5c3a56af-364f-4e10-b1fd-999c985d527f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"
Jul 10 00:49:51.929639 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.929611    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c3a56af-364f-4e10-b1fd-999c985d527f-maas-api-tls\") pod \"maas-api-e2e-inf-a-b8817a-779659c584-jggpp\" (UID: \"5c3a56af-364f-4e10-b1fd-999c985d527f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"
Jul 10 00:49:51.929930 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:49:51.929744    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-serving-cert: secret "maas-api-e2e-inf-a-b8817a-serving-cert" not found
Jul 10 00:49:51.929930 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:49:51.929797    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c3a56af-364f-4e10-b1fd-999c985d527f-maas-api-tls podName:5c3a56af-364f-4e10-b1fd-999c985d527f nodeName:}" failed. No retries permitted until 2026-07-10 00:49:52.429781548 +0000 UTC m=+2042.132566374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5c3a56af-364f-4e10-b1fd-999c985d527f-maas-api-tls") pod "maas-api-e2e-inf-a-b8817a-779659c584-jggpp" (UID: "5c3a56af-364f-4e10-b1fd-999c985d527f") : secret "maas-api-e2e-inf-a-b8817a-serving-cert" not found
Jul 10 00:49:51.930216 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.930194    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c3a56af-364f-4e10-b1fd-999c985d527f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b8817a-779659c584-jggpp\" (UID: \"5c3a56af-364f-4e10-b1fd-999c985d527f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"
Jul 10 00:49:51.938859 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:51.938832    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t9k8\" (UniqueName: \"kubernetes.io/projected/5c3a56af-364f-4e10-b1fd-999c985d527f-kube-api-access-5t9k8\") pod \"maas-api-e2e-inf-a-b8817a-779659c584-jggpp\" (UID: \"5c3a56af-364f-4e10-b1fd-999c985d527f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"
Jul 10 00:49:52.016015 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:52.015967    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:52.223976 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:52.223941    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:52.325227 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:52.325172    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:52.434141 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:52.434109    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c3a56af-364f-4e10-b1fd-999c985d527f-maas-api-tls\") pod \"maas-api-e2e-inf-a-b8817a-779659c584-jggpp\" (UID: \"5c3a56af-364f-4e10-b1fd-999c985d527f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"
Jul 10 00:49:52.436526 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:52.436505    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c3a56af-364f-4e10-b1fd-999c985d527f-maas-api-tls\") pod \"maas-api-e2e-inf-a-b8817a-779659c584-jggpp\" (UID: \"5c3a56af-364f-4e10-b1fd-999c985d527f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"
Jul 10 00:49:52.660421 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:52.660306    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"
Jul 10 00:49:52.793969 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:52.793941    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"]
Jul 10 00:49:52.796446 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:49:52.796419    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c3a56af_364f_4e10_b1fd_999c985d527f.slice/crio-f34c3be83d7b7bf183658516321a2a3c65bb9de0ea7a1e1822ea8b3e85da9f3f WatchSource:0}: Error finding container f34c3be83d7b7bf183658516321a2a3c65bb9de0ea7a1e1822ea8b3e85da9f3f: Status 404 returned error can't find the container with id f34c3be83d7b7bf183658516321a2a3c65bb9de0ea7a1e1822ea8b3e85da9f3f
Jul 10 00:49:52.798257 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:52.798236    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 00:49:53.779236 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:53.779199    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp" event={"ID":"5c3a56af-364f-4e10-b1fd-999c985d527f","Type":"ContainerStarted","Data":"9b2e40e676bba0322712e0a6385def84f4c18dcbce752bfdb8edb0575db9a177"}
Jul 10 00:49:53.779428 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:53.779250    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp" event={"ID":"5c3a56af-364f-4e10-b1fd-999c985d527f","Type":"ContainerStarted","Data":"f34c3be83d7b7bf183658516321a2a3c65bb9de0ea7a1e1822ea8b3e85da9f3f"}
Jul 10 00:49:53.779428 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:53.779292    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"
Jul 10 00:49:53.796106 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:53.796052    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp" podStartSLOduration=2.249464313 podStartE2EDuration="2.796033793s" totalImagesPullingTime="546.56948ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:49:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:49:53.794720464 +0000 UTC m=+2043.497505331" watchObservedRunningTime="2026-07-10 00:49:53.796033793 +0000 UTC m=+2043.498818644"
Jul 10 00:49:56.215855 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:56.215815    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:56.586913 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:56.586818    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:56.914060 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:56.913971    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:57.031338 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:57.031286    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:59.697936 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:59.697903    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:59.788474 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:59.788445    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"
Jul 10 00:49:59.802619 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:59.802585    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:49:59.889946 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:59.889914    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"]
Jul 10 00:49:59.893626 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:59.893601    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"
Jul 10 00:49:59.896215 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:59.896192    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-f86ddd-serving-cert\""
Jul 10 00:49:59.902645 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:59.901691    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"]
Jul 10 00:49:59.924583 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:49:59.924550    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:50:00.008377 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.008267    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c8018aa-04e8-4b5c-a060-d5e33d382a98-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj\" (UID: \"7c8018aa-04e8-4b5c-a060-d5e33d382a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"
Jul 10 00:50:00.008525 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.008435    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dzn6\" (UniqueName: \"kubernetes.io/projected/7c8018aa-04e8-4b5c-a060-d5e33d382a98-kube-api-access-6dzn6\") pod \"maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj\" (UID: \"7c8018aa-04e8-4b5c-a060-d5e33d382a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"
Jul 10 00:50:00.008567 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.008546    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c8018aa-04e8-4b5c-a060-d5e33d382a98-maas-api-tls\") pod \"maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj\" (UID: \"7c8018aa-04e8-4b5c-a060-d5e33d382a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"
Jul 10 00:50:00.074376 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.074325    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:50:00.109589 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.109547    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c8018aa-04e8-4b5c-a060-d5e33d382a98-maas-api-tls\") pod \"maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj\" (UID: \"7c8018aa-04e8-4b5c-a060-d5e33d382a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"
Jul 10 00:50:00.109759 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.109609    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c8018aa-04e8-4b5c-a060-d5e33d382a98-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj\" (UID: \"7c8018aa-04e8-4b5c-a060-d5e33d382a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"
Jul 10 00:50:00.109759 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.109651    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dzn6\" (UniqueName: \"kubernetes.io/projected/7c8018aa-04e8-4b5c-a060-d5e33d382a98-kube-api-access-6dzn6\") pod \"maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj\" (UID: \"7c8018aa-04e8-4b5c-a060-d5e33d382a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"
Jul 10 00:50:00.110371 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.110349    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c8018aa-04e8-4b5c-a060-d5e33d382a98-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj\" (UID: \"7c8018aa-04e8-4b5c-a060-d5e33d382a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"
Jul 10 00:50:00.112036 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.112013    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c8018aa-04e8-4b5c-a060-d5e33d382a98-maas-api-tls\") pod \"maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj\" (UID: \"7c8018aa-04e8-4b5c-a060-d5e33d382a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"
Jul 10 00:50:00.124693 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.124666    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dzn6\" (UniqueName: \"kubernetes.io/projected/7c8018aa-04e8-4b5c-a060-d5e33d382a98-kube-api-access-6dzn6\") pod \"maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj\" (UID: \"7c8018aa-04e8-4b5c-a060-d5e33d382a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"
Jul 10 00:50:00.207803 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.207771    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"
Jul 10 00:50:00.330171 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.330137    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:50:00.343106 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.343085    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"]
Jul 10 00:50:00.345463 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:50:00.345437    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c8018aa_04e8_4b5c_a060_d5e33d382a98.slice/crio-55d8612867b7953e78f3b47fa65a947974fd7a4c6da52f54ce189682f0599a46 WatchSource:0}: Error finding container 55d8612867b7953e78f3b47fa65a947974fd7a4c6da52f54ce189682f0599a46: Status 404 returned error can't find the container with id 55d8612867b7953e78f3b47fa65a947974fd7a4c6da52f54ce189682f0599a46
Jul 10 00:50:00.431819 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.431790    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:50:00.812011 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.811980    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj" event={"ID":"7c8018aa-04e8-4b5c-a060-d5e33d382a98","Type":"ContainerStarted","Data":"95187ba0c1a7b6584cec2b9555cc55817abd5e723a4a0d4765aed59e3e4e3759"}
Jul 10 00:50:00.812411 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.812071    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"
Jul 10 00:50:00.812411 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.812098    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj" event={"ID":"7c8018aa-04e8-4b5c-a060-d5e33d382a98","Type":"ContainerStarted","Data":"55d8612867b7953e78f3b47fa65a947974fd7a4c6da52f54ce189682f0599a46"}
Jul 10 00:50:00.833294 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:00.833241    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj" podStartSLOduration=1.457748767 podStartE2EDuration="1.833222441s" totalImagesPullingTime="375.473674ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:49:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:50:00.832133782 +0000 UTC m=+2050.534918647" watchObservedRunningTime="2026-07-10 00:50:00.833222441 +0000 UTC m=+2050.536007292"
Jul 10 00:50:04.231272 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:04.231234    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:50:06.822765 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:06.822735    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"
Jul 10 00:50:13.759441 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:13.759402    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:50:13.858376 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:13.858342    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:50:20.999062 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:20.999025    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:50:21.101287 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:21.101243    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:50:23.120532 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:23.120496    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:50:50.865470 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:50.865441    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:50:50.870895 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:50:50.870874    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:51:03.419215 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:03.419176    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:51:23.382077 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:23.382037    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:51:24.248309 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:24.248271    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:51:24.418032 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:24.418001    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:51:24.831957 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:24.831899    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:51:24.938485 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:24.938433    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:51:25.052495 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:25.052454    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"]
Jul 10 00:51:25.052748 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:25.052718    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp" podUID="5c3a56af-364f-4e10-b1fd-999c985d527f" containerName="maas-api" containerID="cri-o://9b2e40e676bba0322712e0a6385def84f4c18dcbce752bfdb8edb0575db9a177" gracePeriod=30
Jul 10 00:51:25.071737 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:25.071709    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:51:25.191033 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:25.191001    2559 generic.go:356] "Generic (PLEG): container finished" podID="5c3a56af-364f-4e10-b1fd-999c985d527f" containerID="9b2e40e676bba0322712e0a6385def84f4c18dcbce752bfdb8edb0575db9a177" exitCode=0
Jul 10 00:51:25.191180 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:25.191064    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp" event={"ID":"5c3a56af-364f-4e10-b1fd-999c985d527f","Type":"ContainerDied","Data":"9b2e40e676bba0322712e0a6385def84f4c18dcbce752bfdb8edb0575db9a177"}
Jul 10 00:51:25.301923 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:25.301892    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"
Jul 10 00:51:25.474005 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:25.473982    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5t9k8\" (UniqueName: \"kubernetes.io/projected/5c3a56af-364f-4e10-b1fd-999c985d527f-kube-api-access-5t9k8\") pod \"5c3a56af-364f-4e10-b1fd-999c985d527f\" (UID: \"5c3a56af-364f-4e10-b1fd-999c985d527f\") "
Jul 10 00:51:25.474461 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:25.474021    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c3a56af-364f-4e10-b1fd-999c985d527f-openshift-service-ca\") pod \"5c3a56af-364f-4e10-b1fd-999c985d527f\" (UID: \"5c3a56af-364f-4e10-b1fd-999c985d527f\") "
Jul 10 00:51:25.474461 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:25.474070    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c3a56af-364f-4e10-b1fd-999c985d527f-maas-api-tls\") pod \"5c3a56af-364f-4e10-b1fd-999c985d527f\" (UID: \"5c3a56af-364f-4e10-b1fd-999c985d527f\") "
Jul 10 00:51:25.474461 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:25.474394    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c3a56af-364f-4e10-b1fd-999c985d527f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5c3a56af-364f-4e10-b1fd-999c985d527f" (UID: "5c3a56af-364f-4e10-b1fd-999c985d527f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:51:25.476059 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:25.476034    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c3a56af-364f-4e10-b1fd-999c985d527f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5c3a56af-364f-4e10-b1fd-999c985d527f" (UID: "5c3a56af-364f-4e10-b1fd-999c985d527f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:51:25.476162 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:25.476068    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c3a56af-364f-4e10-b1fd-999c985d527f-kube-api-access-5t9k8" (OuterVolumeSpecName: "kube-api-access-5t9k8") pod "5c3a56af-364f-4e10-b1fd-999c985d527f" (UID: "5c3a56af-364f-4e10-b1fd-999c985d527f"). InnerVolumeSpecName "kube-api-access-5t9k8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:51:25.574798 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:25.574754    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t9k8\" (UniqueName: \"kubernetes.io/projected/5c3a56af-364f-4e10-b1fd-999c985d527f-kube-api-access-5t9k8\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:51:25.574798 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:25.574794    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c3a56af-364f-4e10-b1fd-999c985d527f-openshift-service-ca\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:51:25.574798 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:25.574804    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c3a56af-364f-4e10-b1fd-999c985d527f-maas-api-tls\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:51:26.197480 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:26.197448    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"
Jul 10 00:51:26.197480 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:26.197459    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp" event={"ID":"5c3a56af-364f-4e10-b1fd-999c985d527f","Type":"ContainerDied","Data":"f34c3be83d7b7bf183658516321a2a3c65bb9de0ea7a1e1822ea8b3e85da9f3f"}
Jul 10 00:51:26.197761 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:26.197523    2559 scope.go:117] "RemoveContainer" containerID="9b2e40e676bba0322712e0a6385def84f4c18dcbce752bfdb8edb0575db9a177"
Jul 10 00:51:26.218061 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:26.218032    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"]
Jul 10 00:51:26.220341 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:26.220301    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b8817a-779659c584-jggpp"]
Jul 10 00:51:26.836383 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:26.836349    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c3a56af-364f-4e10-b1fd-999c985d527f" path="/var/lib/kubelet/pods/5c3a56af-364f-4e10-b1fd-999c985d527f/volumes"
Jul 10 00:51:29.914460 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:29.914430    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:51:37.509608 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:51:37.509559    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:06.536274 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:06.536222    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:06.807917 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:06.807804    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:07.053524 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:07.053485    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:07.211972 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:07.211948    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:07.645719 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:07.645617    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:07.846600 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:07.846562    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:07.947189 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:07.947151    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:08.057407 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.057377    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"]
Jul 10 00:52:08.057633 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.057609    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj" podUID="7c8018aa-04e8-4b5c-a060-d5e33d382a98" containerName="maas-api" containerID="cri-o://95187ba0c1a7b6584cec2b9555cc55817abd5e723a4a0d4765aed59e3e4e3759" gracePeriod=30
Jul 10 00:52:08.082348 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.082305    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:08.306604 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.306580    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"
Jul 10 00:52:08.390574 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.390543    2559 generic.go:356] "Generic (PLEG): container finished" podID="7c8018aa-04e8-4b5c-a060-d5e33d382a98" containerID="95187ba0c1a7b6584cec2b9555cc55817abd5e723a4a0d4765aed59e3e4e3759" exitCode=0
Jul 10 00:52:08.390769 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.390606    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"
Jul 10 00:52:08.390769 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.390634    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj" event={"ID":"7c8018aa-04e8-4b5c-a060-d5e33d382a98","Type":"ContainerDied","Data":"95187ba0c1a7b6584cec2b9555cc55817abd5e723a4a0d4765aed59e3e4e3759"}
Jul 10 00:52:08.390769 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.390699    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj" event={"ID":"7c8018aa-04e8-4b5c-a060-d5e33d382a98","Type":"ContainerDied","Data":"55d8612867b7953e78f3b47fa65a947974fd7a4c6da52f54ce189682f0599a46"}
Jul 10 00:52:08.390769 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.390730    2559 scope.go:117] "RemoveContainer" containerID="95187ba0c1a7b6584cec2b9555cc55817abd5e723a4a0d4765aed59e3e4e3759"
Jul 10 00:52:08.400356 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.400340    2559 scope.go:117] "RemoveContainer" containerID="95187ba0c1a7b6584cec2b9555cc55817abd5e723a4a0d4765aed59e3e4e3759"
Jul 10 00:52:08.400602 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:52:08.400582    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95187ba0c1a7b6584cec2b9555cc55817abd5e723a4a0d4765aed59e3e4e3759\": container with ID starting with 95187ba0c1a7b6584cec2b9555cc55817abd5e723a4a0d4765aed59e3e4e3759 not found: ID does not exist" containerID="95187ba0c1a7b6584cec2b9555cc55817abd5e723a4a0d4765aed59e3e4e3759"
Jul 10 00:52:08.400659 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.400612    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95187ba0c1a7b6584cec2b9555cc55817abd5e723a4a0d4765aed59e3e4e3759"} err="failed to get container status \"95187ba0c1a7b6584cec2b9555cc55817abd5e723a4a0d4765aed59e3e4e3759\": rpc error: code = NotFound desc = could not find container \"95187ba0c1a7b6584cec2b9555cc55817abd5e723a4a0d4765aed59e3e4e3759\": container with ID starting with 95187ba0c1a7b6584cec2b9555cc55817abd5e723a4a0d4765aed59e3e4e3759 not found: ID does not exist"
Jul 10 00:52:08.441961 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.441925    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dzn6\" (UniqueName: \"kubernetes.io/projected/7c8018aa-04e8-4b5c-a060-d5e33d382a98-kube-api-access-6dzn6\") pod \"7c8018aa-04e8-4b5c-a060-d5e33d382a98\" (UID: \"7c8018aa-04e8-4b5c-a060-d5e33d382a98\") "
Jul 10 00:52:08.442156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.442032    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c8018aa-04e8-4b5c-a060-d5e33d382a98-openshift-service-ca\") pod \"7c8018aa-04e8-4b5c-a060-d5e33d382a98\" (UID: \"7c8018aa-04e8-4b5c-a060-d5e33d382a98\") "
Jul 10 00:52:08.442156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.442049    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c8018aa-04e8-4b5c-a060-d5e33d382a98-maas-api-tls\") pod \"7c8018aa-04e8-4b5c-a060-d5e33d382a98\" (UID: \"7c8018aa-04e8-4b5c-a060-d5e33d382a98\") "
Jul 10 00:52:08.442499 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.442473    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c8018aa-04e8-4b5c-a060-d5e33d382a98-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7c8018aa-04e8-4b5c-a060-d5e33d382a98" (UID: "7c8018aa-04e8-4b5c-a060-d5e33d382a98"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:52:08.444105 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.444079    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c8018aa-04e8-4b5c-a060-d5e33d382a98-kube-api-access-6dzn6" (OuterVolumeSpecName: "kube-api-access-6dzn6") pod "7c8018aa-04e8-4b5c-a060-d5e33d382a98" (UID: "7c8018aa-04e8-4b5c-a060-d5e33d382a98"). InnerVolumeSpecName "kube-api-access-6dzn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:52:08.444105 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.444089    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c8018aa-04e8-4b5c-a060-d5e33d382a98-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7c8018aa-04e8-4b5c-a060-d5e33d382a98" (UID: "7c8018aa-04e8-4b5c-a060-d5e33d382a98"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:52:08.543457 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.543364    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c8018aa-04e8-4b5c-a060-d5e33d382a98-openshift-service-ca\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:52:08.543457 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.543398    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c8018aa-04e8-4b5c-a060-d5e33d382a98-maas-api-tls\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:52:08.543457 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.543407    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dzn6\" (UniqueName: \"kubernetes.io/projected/7c8018aa-04e8-4b5c-a060-d5e33d382a98-kube-api-access-6dzn6\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:52:08.712528 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.712477    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"]
Jul 10 00:52:08.714574 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.714552    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f86ddd-f4448d674-rdjkj"]
Jul 10 00:52:08.836252 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:08.836177    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c8018aa-04e8-4b5c-a060-d5e33d382a98" path="/var/lib/kubelet/pods/7c8018aa-04e8-4b5c-a060-d5e33d382a98/volumes"
Jul 10 00:52:13.315157 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:13.315114    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:13.418013 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:13.417963    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:13.582487 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:13.582395    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:13.809496 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:13.809462    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:14.164905 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:14.164869    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:14.462608 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:14.462576    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:14.838037 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:14.837947    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:17.435500 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.435446    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:17.505903 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.505866    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"]
Jul 10 00:52:17.506437 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.506421    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c3a56af-364f-4e10-b1fd-999c985d527f" containerName="maas-api"
Jul 10 00:52:17.506491 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.506441    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c3a56af-364f-4e10-b1fd-999c985d527f" containerName="maas-api"
Jul 10 00:52:17.506491 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.506456    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c8018aa-04e8-4b5c-a060-d5e33d382a98" containerName="maas-api"
Jul 10 00:52:17.506491 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.506465    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c8018aa-04e8-4b5c-a060-d5e33d382a98" containerName="maas-api"
Jul 10 00:52:17.506588 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.506545    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c3a56af-364f-4e10-b1fd-999c985d527f" containerName="maas-api"
Jul 10 00:52:17.506588 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.506564    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c8018aa-04e8-4b5c-a060-d5e33d382a98" containerName="maas-api"
Jul 10 00:52:17.508619 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.508587    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:17.511586 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.511446    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"
Jul 10 00:52:17.514243 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.514219    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-f7d727-serving-cert\""
Jul 10 00:52:17.514376 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.514219    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vpn7x\""
Jul 10 00:52:17.530184 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.530160    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"]
Jul 10 00:52:17.575683 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.575649    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:17.627566 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.627531    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hhzp\" (UniqueName: \"kubernetes.io/projected/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-kube-api-access-8hhzp\") pod \"maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb\" (UID: \"9396b47a-9a57-40f8-89e1-9ff23a2d9df0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"
Jul 10 00:52:17.627748 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.627579    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-openshift-service-ca\") pod \"maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb\" (UID: \"9396b47a-9a57-40f8-89e1-9ff23a2d9df0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"
Jul 10 00:52:17.627748 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.627649    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-maas-api-tls\") pod \"maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb\" (UID: \"9396b47a-9a57-40f8-89e1-9ff23a2d9df0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"
Jul 10 00:52:17.675156 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.675122    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:17.728303 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.728257    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-maas-api-tls\") pod \"maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb\" (UID: \"9396b47a-9a57-40f8-89e1-9ff23a2d9df0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"
Jul 10 00:52:17.728518 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.728346    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hhzp\" (UniqueName: \"kubernetes.io/projected/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-kube-api-access-8hhzp\") pod \"maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb\" (UID: \"9396b47a-9a57-40f8-89e1-9ff23a2d9df0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"
Jul 10 00:52:17.728518 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.728378    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-openshift-service-ca\") pod \"maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb\" (UID: \"9396b47a-9a57-40f8-89e1-9ff23a2d9df0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"
Jul 10 00:52:17.728998 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.728979    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-openshift-service-ca\") pod \"maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb\" (UID: \"9396b47a-9a57-40f8-89e1-9ff23a2d9df0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"
Jul 10 00:52:17.730734 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.730714    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-maas-api-tls\") pod \"maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb\" (UID: \"9396b47a-9a57-40f8-89e1-9ff23a2d9df0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"
Jul 10 00:52:17.736718 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.736694    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hhzp\" (UniqueName: \"kubernetes.io/projected/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-kube-api-access-8hhzp\") pod \"maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb\" (UID: \"9396b47a-9a57-40f8-89e1-9ff23a2d9df0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"
Jul 10 00:52:17.832507 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.832470    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"
Jul 10 00:52:17.964054 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.963979    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"]
Jul 10 00:52:17.966752 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:52:17.966722    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9396b47a_9a57_40f8_89e1_9ff23a2d9df0.slice/crio-ba36b2d09c8ef4c48ad579ecb83548b6940f60ccaa19dd89e54ebc9c15d3674e WatchSource:0}: Error finding container ba36b2d09c8ef4c48ad579ecb83548b6940f60ccaa19dd89e54ebc9c15d3674e: Status 404 returned error can't find the container with id ba36b2d09c8ef4c48ad579ecb83548b6940f60ccaa19dd89e54ebc9c15d3674e
Jul 10 00:52:17.968365 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:17.968241    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:18.024706 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:18.024615    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:18.444067 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:18.444028    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb" event={"ID":"9396b47a-9a57-40f8-89e1-9ff23a2d9df0","Type":"ContainerStarted","Data":"dee41c8b1343d585269137f9a2305beb371acdc52d217544a75d3f3cb84700d6"}
Jul 10 00:52:18.444463 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:18.444088    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb" event={"ID":"9396b47a-9a57-40f8-89e1-9ff23a2d9df0","Type":"ContainerStarted","Data":"ba36b2d09c8ef4c48ad579ecb83548b6940f60ccaa19dd89e54ebc9c15d3674e"}
Jul 10 00:52:18.444463 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:18.444210    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"
Jul 10 00:52:18.462148 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:18.462101    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb" podStartSLOduration=1.089207974 podStartE2EDuration="1.46208665s" totalImagesPullingTime="372.878676ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:52:18.460548661 +0000 UTC m=+2188.163333510" watchObservedRunningTime="2026-07-10 00:52:18.46208665 +0000 UTC m=+2188.164871498"
Jul 10 00:52:21.947810 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:21.947775    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:22.391130 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:22.391046    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:22.702253 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:22.702224    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:22.782373 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:22.782341    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:24.453501 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:24.453469    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"
Jul 10 00:52:25.469133 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.469098    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:25.526696 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.526654    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"]
Jul 10 00:52:25.530484 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.530461    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"
Jul 10 00:52:25.532740 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.532719    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-0af175-serving-cert\""
Jul 10 00:52:25.542380 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.542357    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"]
Jul 10 00:52:25.549147 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.549124    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:25.619147 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.619105    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:25.699699 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.699666    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/848c867f-6649-40dc-b320-bea1a409981e-openshift-service-ca\") pod \"maas-api-e2e-api-b-0af175-6b55bf774c-7nd97\" (UID: \"848c867f-6649-40dc-b320-bea1a409981e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"
Jul 10 00:52:25.699855 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.699739    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/848c867f-6649-40dc-b320-bea1a409981e-maas-api-tls\") pod \"maas-api-e2e-api-b-0af175-6b55bf774c-7nd97\" (UID: \"848c867f-6649-40dc-b320-bea1a409981e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"
Jul 10 00:52:25.699855 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.699764    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lsjp\" (UniqueName: \"kubernetes.io/projected/848c867f-6649-40dc-b320-bea1a409981e-kube-api-access-8lsjp\") pod \"maas-api-e2e-api-b-0af175-6b55bf774c-7nd97\" (UID: \"848c867f-6649-40dc-b320-bea1a409981e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"
Jul 10 00:52:25.748674 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.748591    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:25.800446 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.800414    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/848c867f-6649-40dc-b320-bea1a409981e-openshift-service-ca\") pod \"maas-api-e2e-api-b-0af175-6b55bf774c-7nd97\" (UID: \"848c867f-6649-40dc-b320-bea1a409981e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"
Jul 10 00:52:25.800639 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.800488    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/848c867f-6649-40dc-b320-bea1a409981e-maas-api-tls\") pod \"maas-api-e2e-api-b-0af175-6b55bf774c-7nd97\" (UID: \"848c867f-6649-40dc-b320-bea1a409981e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"
Jul 10 00:52:25.800639 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.800518    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lsjp\" (UniqueName: \"kubernetes.io/projected/848c867f-6649-40dc-b320-bea1a409981e-kube-api-access-8lsjp\") pod \"maas-api-e2e-api-b-0af175-6b55bf774c-7nd97\" (UID: \"848c867f-6649-40dc-b320-bea1a409981e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"
Jul 10 00:52:25.801122 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.801096    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/848c867f-6649-40dc-b320-bea1a409981e-openshift-service-ca\") pod \"maas-api-e2e-api-b-0af175-6b55bf774c-7nd97\" (UID: \"848c867f-6649-40dc-b320-bea1a409981e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"
Jul 10 00:52:25.803044 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.803017    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/848c867f-6649-40dc-b320-bea1a409981e-maas-api-tls\") pod \"maas-api-e2e-api-b-0af175-6b55bf774c-7nd97\" (UID: \"848c867f-6649-40dc-b320-bea1a409981e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"
Jul 10 00:52:25.808760 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.808734    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lsjp\" (UniqueName: \"kubernetes.io/projected/848c867f-6649-40dc-b320-bea1a409981e-kube-api-access-8lsjp\") pod \"maas-api-e2e-api-b-0af175-6b55bf774c-7nd97\" (UID: \"848c867f-6649-40dc-b320-bea1a409981e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"
Jul 10 00:52:25.844791 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.844769    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"
Jul 10 00:52:25.983239 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.983214    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"]
Jul 10 00:52:25.985676 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:52:25.985654    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod848c867f_6649_40dc_b320_bea1a409981e.slice/crio-7ae1b7844a5fc6016f751af8f94514ce22664c4591a790d0e3bc492185e488f4 WatchSource:0}: Error finding container 7ae1b7844a5fc6016f751af8f94514ce22664c4591a790d0e3bc492185e488f4: Status 404 returned error can't find the container with id 7ae1b7844a5fc6016f751af8f94514ce22664c4591a790d0e3bc492185e488f4
Jul 10 00:52:25.998537 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:25.998507    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:26.099539 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:26.099421    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:26.481563 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:26.481530    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97" event={"ID":"848c867f-6649-40dc-b320-bea1a409981e","Type":"ContainerStarted","Data":"7ae1b7844a5fc6016f751af8f94514ce22664c4591a790d0e3bc492185e488f4"}
Jul 10 00:52:27.487636 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:27.487600    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97" event={"ID":"848c867f-6649-40dc-b320-bea1a409981e","Type":"ContainerStarted","Data":"8f297d3fc6f2bffd7f9d2df467317862e0c06305a72cb4f54123e1df4c92a107"}
Jul 10 00:52:27.488060 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:27.487740    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"
Jul 10 00:52:27.509041 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:27.508987    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97" podStartSLOduration=1.959350892 podStartE2EDuration="2.508971524s" totalImagesPullingTime="549.620632ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:52:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:52:27.506556836 +0000 UTC m=+2197.209341688" watchObservedRunningTime="2026-07-10 00:52:27.508971524 +0000 UTC m=+2197.211756370"
Jul 10 00:52:28.817067 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:28.817037    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:28.914741 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:28.914698    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:29.968478 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:29.968445    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:33.499261 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:33.499230    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"
Jul 10 00:52:38.287169 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:38.287135    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:38.678266 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:38.678168    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:38.881125 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:38.881090    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:38.979754 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:38.979725    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:39.152677 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.152647    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"]
Jul 10 00:52:39.152865 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.152838    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb" podUID="9396b47a-9a57-40f8-89e1-9ff23a2d9df0" containerName="maas-api" containerID="cri-o://dee41c8b1343d585269137f9a2305beb371acdc52d217544a75d3f3cb84700d6" gracePeriod=30
Jul 10 00:52:39.190851 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.190816    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:39.398911 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.398883    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"
Jul 10 00:52:39.411850 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.411826    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-openshift-service-ca\") pod \"9396b47a-9a57-40f8-89e1-9ff23a2d9df0\" (UID: \"9396b47a-9a57-40f8-89e1-9ff23a2d9df0\") "
Jul 10 00:52:39.411976 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.411911    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-maas-api-tls\") pod \"9396b47a-9a57-40f8-89e1-9ff23a2d9df0\" (UID: \"9396b47a-9a57-40f8-89e1-9ff23a2d9df0\") "
Jul 10 00:52:39.411976 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.411940    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hhzp\" (UniqueName: \"kubernetes.io/projected/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-kube-api-access-8hhzp\") pod \"9396b47a-9a57-40f8-89e1-9ff23a2d9df0\" (UID: \"9396b47a-9a57-40f8-89e1-9ff23a2d9df0\") "
Jul 10 00:52:39.412195 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.412170    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9396b47a-9a57-40f8-89e1-9ff23a2d9df0" (UID: "9396b47a-9a57-40f8-89e1-9ff23a2d9df0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:52:39.413964 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.413937    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-kube-api-access-8hhzp" (OuterVolumeSpecName: "kube-api-access-8hhzp") pod "9396b47a-9a57-40f8-89e1-9ff23a2d9df0" (UID: "9396b47a-9a57-40f8-89e1-9ff23a2d9df0"). InnerVolumeSpecName "kube-api-access-8hhzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:52:39.413964 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.413946    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9396b47a-9a57-40f8-89e1-9ff23a2d9df0" (UID: "9396b47a-9a57-40f8-89e1-9ff23a2d9df0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:52:39.512881 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.512845    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-openshift-service-ca\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:52:39.512881 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.512875    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-maas-api-tls\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:52:39.512881 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.512888    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hhzp\" (UniqueName: \"kubernetes.io/projected/9396b47a-9a57-40f8-89e1-9ff23a2d9df0-kube-api-access-8hhzp\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:52:39.544821 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.544738    2559 generic.go:356] "Generic (PLEG): container finished" podID="9396b47a-9a57-40f8-89e1-9ff23a2d9df0" containerID="dee41c8b1343d585269137f9a2305beb371acdc52d217544a75d3f3cb84700d6" exitCode=0
Jul 10 00:52:39.544821 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.544798    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"
Jul 10 00:52:39.544987 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.544822    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb" event={"ID":"9396b47a-9a57-40f8-89e1-9ff23a2d9df0","Type":"ContainerDied","Data":"dee41c8b1343d585269137f9a2305beb371acdc52d217544a75d3f3cb84700d6"}
Jul 10 00:52:39.544987 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.544881    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb" event={"ID":"9396b47a-9a57-40f8-89e1-9ff23a2d9df0","Type":"ContainerDied","Data":"ba36b2d09c8ef4c48ad579ecb83548b6940f60ccaa19dd89e54ebc9c15d3674e"}
Jul 10 00:52:39.544987 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.544921    2559 scope.go:117] "RemoveContainer" containerID="dee41c8b1343d585269137f9a2305beb371acdc52d217544a75d3f3cb84700d6"
Jul 10 00:52:39.554154 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.554137    2559 scope.go:117] "RemoveContainer" containerID="dee41c8b1343d585269137f9a2305beb371acdc52d217544a75d3f3cb84700d6"
Jul 10 00:52:39.554556 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:52:39.554537    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dee41c8b1343d585269137f9a2305beb371acdc52d217544a75d3f3cb84700d6\": container with ID starting with dee41c8b1343d585269137f9a2305beb371acdc52d217544a75d3f3cb84700d6 not found: ID does not exist" containerID="dee41c8b1343d585269137f9a2305beb371acdc52d217544a75d3f3cb84700d6"
Jul 10 00:52:39.554632 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.554564    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dee41c8b1343d585269137f9a2305beb371acdc52d217544a75d3f3cb84700d6"} err="failed to get container status \"dee41c8b1343d585269137f9a2305beb371acdc52d217544a75d3f3cb84700d6\": rpc error: code = NotFound desc = could not find container \"dee41c8b1343d585269137f9a2305beb371acdc52d217544a75d3f3cb84700d6\": container with ID starting with dee41c8b1343d585269137f9a2305beb371acdc52d217544a75d3f3cb84700d6 not found: ID does not exist"
Jul 10 00:52:39.569074 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.569051    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"]
Jul 10 00:52:39.576356 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:39.576308    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f7d727-d6766ccc5-58sbb"]
Jul 10 00:52:40.838328 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:40.838293    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9396b47a-9a57-40f8-89e1-9ff23a2d9df0" path="/var/lib/kubelet/pods/9396b47a-9a57-40f8-89e1-9ff23a2d9df0/volumes"
Jul 10 00:52:44.575776 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:44.575739    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:44.675839 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:44.675804    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:44.842647 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:44.842558    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:45.068393 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:45.068356    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:45.226510 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:45.226474    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:45.648728 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:45.648635    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:45.810758 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:45.810711    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:45.914483 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:45.914399    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:46.042850 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.042818    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"]
Jul 10 00:52:46.043108 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.043076    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97" podUID="848c867f-6649-40dc-b320-bea1a409981e" containerName="maas-api" containerID="cri-o://8f297d3fc6f2bffd7f9d2df467317862e0c06305a72cb4f54123e1df4c92a107" gracePeriod=30
Jul 10 00:52:46.066371 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.066347    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:46.290972 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.290948    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"
Jul 10 00:52:46.375646 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.375609    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/848c867f-6649-40dc-b320-bea1a409981e-openshift-service-ca\") pod \"848c867f-6649-40dc-b320-bea1a409981e\" (UID: \"848c867f-6649-40dc-b320-bea1a409981e\") "
Jul 10 00:52:46.375646 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.375648    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/848c867f-6649-40dc-b320-bea1a409981e-maas-api-tls\") pod \"848c867f-6649-40dc-b320-bea1a409981e\" (UID: \"848c867f-6649-40dc-b320-bea1a409981e\") "
Jul 10 00:52:46.375915 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.375700    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lsjp\" (UniqueName: \"kubernetes.io/projected/848c867f-6649-40dc-b320-bea1a409981e-kube-api-access-8lsjp\") pod \"848c867f-6649-40dc-b320-bea1a409981e\" (UID: \"848c867f-6649-40dc-b320-bea1a409981e\") "
Jul 10 00:52:46.376035 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.376013    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/848c867f-6649-40dc-b320-bea1a409981e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "848c867f-6649-40dc-b320-bea1a409981e" (UID: "848c867f-6649-40dc-b320-bea1a409981e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:52:46.377819 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.377786    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/848c867f-6649-40dc-b320-bea1a409981e-kube-api-access-8lsjp" (OuterVolumeSpecName: "kube-api-access-8lsjp") pod "848c867f-6649-40dc-b320-bea1a409981e" (UID: "848c867f-6649-40dc-b320-bea1a409981e"). InnerVolumeSpecName "kube-api-access-8lsjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:52:46.377819 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.377788    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/848c867f-6649-40dc-b320-bea1a409981e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "848c867f-6649-40dc-b320-bea1a409981e" (UID: "848c867f-6649-40dc-b320-bea1a409981e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:52:46.476450 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.476347    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/848c867f-6649-40dc-b320-bea1a409981e-openshift-service-ca\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:52:46.476450 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.476382    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/848c867f-6649-40dc-b320-bea1a409981e-maas-api-tls\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:52:46.476450 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.476397    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lsjp\" (UniqueName: \"kubernetes.io/projected/848c867f-6649-40dc-b320-bea1a409981e-kube-api-access-8lsjp\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:52:46.580167 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.580131    2559 generic.go:356] "Generic (PLEG): container finished" podID="848c867f-6649-40dc-b320-bea1a409981e" containerID="8f297d3fc6f2bffd7f9d2df467317862e0c06305a72cb4f54123e1df4c92a107" exitCode=0
Jul 10 00:52:46.580368 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.580177    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"
Jul 10 00:52:46.580368 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.580200    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97" event={"ID":"848c867f-6649-40dc-b320-bea1a409981e","Type":"ContainerDied","Data":"8f297d3fc6f2bffd7f9d2df467317862e0c06305a72cb4f54123e1df4c92a107"}
Jul 10 00:52:46.580368 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.580243    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97" event={"ID":"848c867f-6649-40dc-b320-bea1a409981e","Type":"ContainerDied","Data":"7ae1b7844a5fc6016f751af8f94514ce22664c4591a790d0e3bc492185e488f4"}
Jul 10 00:52:46.580368 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.580266    2559 scope.go:117] "RemoveContainer" containerID="8f297d3fc6f2bffd7f9d2df467317862e0c06305a72cb4f54123e1df4c92a107"
Jul 10 00:52:46.588959 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.588932    2559 scope.go:117] "RemoveContainer" containerID="8f297d3fc6f2bffd7f9d2df467317862e0c06305a72cb4f54123e1df4c92a107"
Jul 10 00:52:46.589255 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:52:46.589234    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f297d3fc6f2bffd7f9d2df467317862e0c06305a72cb4f54123e1df4c92a107\": container with ID starting with 8f297d3fc6f2bffd7f9d2df467317862e0c06305a72cb4f54123e1df4c92a107 not found: ID does not exist" containerID="8f297d3fc6f2bffd7f9d2df467317862e0c06305a72cb4f54123e1df4c92a107"
Jul 10 00:52:46.589363 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.589267    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f297d3fc6f2bffd7f9d2df467317862e0c06305a72cb4f54123e1df4c92a107"} err="failed to get container status \"8f297d3fc6f2bffd7f9d2df467317862e0c06305a72cb4f54123e1df4c92a107\": rpc error: code = NotFound desc = could not find container \"8f297d3fc6f2bffd7f9d2df467317862e0c06305a72cb4f54123e1df4c92a107\": container with ID starting with 8f297d3fc6f2bffd7f9d2df467317862e0c06305a72cb4f54123e1df4c92a107 not found: ID does not exist"
Jul 10 00:52:46.604638 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.604612    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"]
Jul 10 00:52:46.606175 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.606152    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0af175-6b55bf774c-7nd97"]
Jul 10 00:52:46.836777 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:46.836702    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="848c867f-6649-40dc-b320-bea1a409981e" path="/var/lib/kubelet/pods/848c867f-6649-40dc-b320-bea1a409981e/volumes"
Jul 10 00:52:51.253475 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:51.253385    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:51.307845 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:51.307800    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:51.524237 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:51.524127    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:51.844951 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:51.844852    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:52.138259 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:52.138166    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:52.444555 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:52.444526    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:52.555957 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:52.555926    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:55.651443 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.651405    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:55.731054 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.731016    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:55.731637 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.731615    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"]
Jul 10 00:52:55.732150 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.732131    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="848c867f-6649-40dc-b320-bea1a409981e" containerName="maas-api"
Jul 10 00:52:55.732150 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.732152    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="848c867f-6649-40dc-b320-bea1a409981e" containerName="maas-api"
Jul 10 00:52:55.732287 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.732166    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9396b47a-9a57-40f8-89e1-9ff23a2d9df0" containerName="maas-api"
Jul 10 00:52:55.732287 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.732172    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9396b47a-9a57-40f8-89e1-9ff23a2d9df0" containerName="maas-api"
Jul 10 00:52:55.732287 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.732242    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="848c867f-6649-40dc-b320-bea1a409981e" containerName="maas-api"
Jul 10 00:52:55.732287 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.732252    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9396b47a-9a57-40f8-89e1-9ff23a2d9df0" containerName="maas-api"
Jul 10 00:52:55.735218 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.735201    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"
Jul 10 00:52:55.737793 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.737770    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-d62432-serving-cert\""
Jul 10 00:52:55.737888 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.737780    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vpn7x\""
Jul 10 00:52:55.742809 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.742787    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"]
Jul 10 00:52:55.796146 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.796116    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:55.860978 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.860944    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z\" (UID: \"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"
Jul 10 00:52:55.861144 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.861011    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbcr6\" (UniqueName: \"kubernetes.io/projected/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-kube-api-access-kbcr6\") pod \"maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z\" (UID: \"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"
Jul 10 00:52:55.861144 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.861034    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-maas-api-tls\") pod \"maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z\" (UID: \"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"
Jul 10 00:52:55.940276 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.940186    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:55.962390 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.962366    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z\" (UID: \"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"
Jul 10 00:52:55.962494 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.962432    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbcr6\" (UniqueName: \"kubernetes.io/projected/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-kube-api-access-kbcr6\") pod \"maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z\" (UID: \"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"
Jul 10 00:52:55.962494 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.962454    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-maas-api-tls\") pod \"maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z\" (UID: \"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"
Jul 10 00:52:55.962990 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.962972    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z\" (UID: \"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"
Jul 10 00:52:55.964837 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.964820    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-maas-api-tls\") pod \"maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z\" (UID: \"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"
Jul 10 00:52:55.970069 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:55.970052    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbcr6\" (UniqueName: \"kubernetes.io/projected/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-kube-api-access-kbcr6\") pod \"maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z\" (UID: \"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"
Jul 10 00:52:56.047505 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:56.047466    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"
Jul 10 00:52:56.181343 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:56.181267    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"]
Jul 10 00:52:56.184123 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:52:56.184097    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcafc18d0_5b62_4c7d_bb06_7a9e9558ebd5.slice/crio-6667b3fde8898cb6a97fa3a7ac10a03d4ff76df8f1cbcb3d2a60f379071a22c1 WatchSource:0}: Error finding container 6667b3fde8898cb6a97fa3a7ac10a03d4ff76df8f1cbcb3d2a60f379071a22c1: Status 404 returned error can't find the container with id 6667b3fde8898cb6a97fa3a7ac10a03d4ff76df8f1cbcb3d2a60f379071a22c1
Jul 10 00:52:56.286342 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:56.286256    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:56.385026 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:56.384989    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:52:56.632228 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:56.632143    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z" event={"ID":"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5","Type":"ContainerStarted","Data":"6667b3fde8898cb6a97fa3a7ac10a03d4ff76df8f1cbcb3d2a60f379071a22c1"}
Jul 10 00:52:57.637901 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:57.637863    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z" event={"ID":"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5","Type":"ContainerStarted","Data":"9db3d7a66c25a6a363bfe9b708879c4f686a62f227c3325008d850804b976944"}
Jul 10 00:52:57.638292 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:57.637979    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"
Jul 10 00:52:57.653628 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:52:57.653556    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z" podStartSLOduration=2.176605031 podStartE2EDuration="2.653541685s" totalImagesPullingTime="476.936654ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:52:57.652808396 +0000 UTC m=+2227.355593255" watchObservedRunningTime="2026-07-10 00:52:57.653541685 +0000 UTC m=+2227.356326534"
Jul 10 00:53:00.181278 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:00.181240    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:00.580185 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:00.580148    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:00.858000 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:00.857911    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:01.666294 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:01.666261    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:03.649358 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:03.649329    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"
Jul 10 00:53:03.711456 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:03.711415    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:03.800496 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:03.800464    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:03.898461 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:03.898430    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"]
Jul 10 00:53:03.901803 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:03.901742    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"
Jul 10 00:53:03.904127 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:03.904108    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-f84011-serving-cert\""
Jul 10 00:53:03.911939 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:03.911913    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"]
Jul 10 00:53:03.936965 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:03.936941    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:04.036711 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:04.036676    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knsjr\" (UniqueName: \"kubernetes.io/projected/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-kube-api-access-knsjr\") pod \"maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk\" (UID: \"a7ec94a9-a22f-4265-aef8-e7036a21cc3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"
Jul 10 00:53:04.036861 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:04.036763    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-maas-api-tls\") pod \"maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk\" (UID: \"a7ec94a9-a22f-4265-aef8-e7036a21cc3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"
Jul 10 00:53:04.036861 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:04.036786    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk\" (UID: \"a7ec94a9-a22f-4265-aef8-e7036a21cc3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"
Jul 10 00:53:04.138083 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:04.138040    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-maas-api-tls\") pod \"maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk\" (UID: \"a7ec94a9-a22f-4265-aef8-e7036a21cc3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"
Jul 10 00:53:04.138286 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:04.138098    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk\" (UID: \"a7ec94a9-a22f-4265-aef8-e7036a21cc3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"
Jul 10 00:53:04.138286 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:04.138171    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knsjr\" (UniqueName: \"kubernetes.io/projected/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-kube-api-access-knsjr\") pod \"maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk\" (UID: \"a7ec94a9-a22f-4265-aef8-e7036a21cc3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"
Jul 10 00:53:04.138286 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:53:04.138202    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-serving-cert: secret "maas-api-e2e-shared-b-f84011-serving-cert" not found
Jul 10 00:53:04.138483 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:53:04.138286    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-maas-api-tls podName:a7ec94a9-a22f-4265-aef8-e7036a21cc3a nodeName:}" failed. No retries permitted until 2026-07-10 00:53:04.638263409 +0000 UTC m=+2234.341048238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-maas-api-tls") pod "maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk" (UID: "a7ec94a9-a22f-4265-aef8-e7036a21cc3a") : secret "maas-api-e2e-shared-b-f84011-serving-cert" not found
Jul 10 00:53:04.138896 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:04.138868    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk\" (UID: \"a7ec94a9-a22f-4265-aef8-e7036a21cc3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"
Jul 10 00:53:04.146474 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:04.146448    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knsjr\" (UniqueName: \"kubernetes.io/projected/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-kube-api-access-knsjr\") pod \"maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk\" (UID: \"a7ec94a9-a22f-4265-aef8-e7036a21cc3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"
Jul 10 00:53:04.328668 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:04.328636    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:04.421499 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:04.421464    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:04.522194 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:04.522160    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:04.644381 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:04.644274    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-maas-api-tls\") pod \"maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk\" (UID: \"a7ec94a9-a22f-4265-aef8-e7036a21cc3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"
Jul 10 00:53:04.646591 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:04.646569    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-maas-api-tls\") pod \"maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk\" (UID: \"a7ec94a9-a22f-4265-aef8-e7036a21cc3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"
Jul 10 00:53:04.815692 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:04.815658    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"
Jul 10 00:53:04.948567 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:04.948471    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"]
Jul 10 00:53:04.951352 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:53:04.951322    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ec94a9_a22f_4265_aef8_e7036a21cc3a.slice/crio-9c3b7354b4abdaea99f104467583c56ebb41a50fecae642571c1b253baa8a1e1 WatchSource:0}: Error finding container 9c3b7354b4abdaea99f104467583c56ebb41a50fecae642571c1b253baa8a1e1: Status 404 returned error can't find the container with id 9c3b7354b4abdaea99f104467583c56ebb41a50fecae642571c1b253baa8a1e1
Jul 10 00:53:05.677120 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:05.677022    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk" event={"ID":"a7ec94a9-a22f-4265-aef8-e7036a21cc3a","Type":"ContainerStarted","Data":"d17d55de0f1b897ce8a6b0dc3c353d6036154d4fd5242e6c9446408ca2904de1"}
Jul 10 00:53:05.677274 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:05.677216    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"
Jul 10 00:53:05.677346 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:05.677268    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk" event={"ID":"a7ec94a9-a22f-4265-aef8-e7036a21cc3a","Type":"ContainerStarted","Data":"9c3b7354b4abdaea99f104467583c56ebb41a50fecae642571c1b253baa8a1e1"}
Jul 10 00:53:05.693893 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:05.693842    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk" podStartSLOduration=2.342089105 podStartE2EDuration="2.69382798s" totalImagesPullingTime="351.738875ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:53:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:53:05.691962548 +0000 UTC m=+2235.394747410" watchObservedRunningTime="2026-07-10 00:53:05.69382798 +0000 UTC m=+2235.396612827"
Jul 10 00:53:08.224749 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:08.224712    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:11.687837 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:11.687808    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"
Jul 10 00:53:16.978989 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:16.978958    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:17.075307 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:17.075272    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:25.026175 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:25.026141    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:25.127142 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:25.127091    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:27.113670 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:27.113638    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:48.120246 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:48.120214    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:53:49.114168 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:53:49.114128    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:54:01.026532 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:54:01.026498    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:55:39.240017 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:55:39.239979    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:55:40.094688 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:55:40.094656    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:55:44.010384 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:55:44.010350    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:55:50.893957 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:55:50.893925    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:55:50.911513 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:55:50.911485    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:56:06.790298 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:56:06.790250    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:56:10.811466 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:56:10.811424    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:56:45.626590 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:56:45.626555    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:56:46.587818 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:56:46.587783    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:56:51.677108 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:56:51.677069    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:57:12.520596 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:57:12.520561    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:57:19.115845 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:57:19.115813    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:57:29.503333 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:57:29.503290    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:57:29.601483 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:57:29.601449    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:57:55.771108 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:57:55.771074    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:08.090874 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:08.090834    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:08.997102 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:08.997073    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:17.705568 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:17.705520    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:17.808841 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:17.808800    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:25.705380 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:25.705341    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:25.806880 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:25.806843    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:27.087806 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:27.087768    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:27.479669 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:27.479634    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:27.580709 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:27.580675    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:27.639617 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:27.639582    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:27.714340 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:27.714296    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"]
Jul 10 00:58:27.714563 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:27.714536    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z" podUID="cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5" containerName="maas-api" containerID="cri-o://9db3d7a66c25a6a363bfe9b708879c4f686a62f227c3325008d850804b976944" gracePeriod=30
Jul 10 00:58:27.737711 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:27.737629    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:28.157753 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:28.157724    2559 generic.go:356] "Generic (PLEG): container finished" podID="cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5" containerID="9db3d7a66c25a6a363bfe9b708879c4f686a62f227c3325008d850804b976944" exitCode=0
Jul 10 00:58:28.158101 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:28.157800    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z" event={"ID":"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5","Type":"ContainerDied","Data":"9db3d7a66c25a6a363bfe9b708879c4f686a62f227c3325008d850804b976944"}
Jul 10 00:58:28.158101 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:28.157868    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z" event={"ID":"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5","Type":"ContainerDied","Data":"6667b3fde8898cb6a97fa3a7ac10a03d4ff76df8f1cbcb3d2a60f379071a22c1"}
Jul 10 00:58:28.158101 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:28.157895    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6667b3fde8898cb6a97fa3a7ac10a03d4ff76df8f1cbcb3d2a60f379071a22c1"
Jul 10 00:58:28.163204 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:28.163184    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"
Jul 10 00:58:28.289200 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:28.289112    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-maas-api-tls\") pod \"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5\" (UID: \"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5\") "
Jul 10 00:58:28.289200 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:28.289163    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-openshift-service-ca\") pod \"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5\" (UID: \"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5\") "
Jul 10 00:58:28.289448 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:28.289288    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbcr6\" (UniqueName: \"kubernetes.io/projected/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-kube-api-access-kbcr6\") pod \"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5\" (UID: \"cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5\") "
Jul 10 00:58:28.289706 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:28.289679    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5" (UID: "cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:58:28.291384 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:28.291362    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5" (UID: "cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:58:28.291384 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:28.291371    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-kube-api-access-kbcr6" (OuterVolumeSpecName: "kube-api-access-kbcr6") pod "cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5" (UID: "cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5"). InnerVolumeSpecName "kube-api-access-kbcr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:58:28.390599 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:28.390559    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbcr6\" (UniqueName: \"kubernetes.io/projected/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-kube-api-access-kbcr6\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:58:28.390599 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:28.390590    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-maas-api-tls\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:58:28.390599 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:28.390600    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5-openshift-service-ca\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:58:29.162804 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:29.162708    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"
Jul 10 00:58:29.178476 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:29.178449    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"]
Jul 10 00:58:29.182168 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:29.182145    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d62432-7989d6878d-9ph4z"]
Jul 10 00:58:30.835958 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:30.835926    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5" path="/var/lib/kubelet/pods/cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5/volumes"
Jul 10 00:58:33.298767 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:33.298725    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:33.399076 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:33.399043    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:33.573167 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:33.573082    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:33.793988 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:33.793952    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:33.950465 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:33.950434    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:34.365186 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:34.365099    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:34.466553 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:34.466520    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:34.586212 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:34.586179    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"]
Jul 10 00:58:34.586458 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:34.586428    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk" podUID="a7ec94a9-a22f-4265-aef8-e7036a21cc3a" containerName="maas-api" containerID="cri-o://d17d55de0f1b897ce8a6b0dc3c353d6036154d4fd5242e6c9446408ca2904de1" gracePeriod=30
Jul 10 00:58:34.609810 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:34.609788    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:34.831619 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:34.831593    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"
Jul 10 00:58:34.950564 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:34.950533    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-maas-api-tls\") pod \"a7ec94a9-a22f-4265-aef8-e7036a21cc3a\" (UID: \"a7ec94a9-a22f-4265-aef8-e7036a21cc3a\") "
Jul 10 00:58:34.950762 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:34.950599    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-openshift-service-ca\") pod \"a7ec94a9-a22f-4265-aef8-e7036a21cc3a\" (UID: \"a7ec94a9-a22f-4265-aef8-e7036a21cc3a\") "
Jul 10 00:58:34.950762 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:34.950649    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knsjr\" (UniqueName: \"kubernetes.io/projected/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-kube-api-access-knsjr\") pod \"a7ec94a9-a22f-4265-aef8-e7036a21cc3a\" (UID: \"a7ec94a9-a22f-4265-aef8-e7036a21cc3a\") "
Jul 10 00:58:34.950964 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:34.950941    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a7ec94a9-a22f-4265-aef8-e7036a21cc3a" (UID: "a7ec94a9-a22f-4265-aef8-e7036a21cc3a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 00:58:34.952581 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:34.952563    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a7ec94a9-a22f-4265-aef8-e7036a21cc3a" (UID: "a7ec94a9-a22f-4265-aef8-e7036a21cc3a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 00:58:34.952659 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:34.952596    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-kube-api-access-knsjr" (OuterVolumeSpecName: "kube-api-access-knsjr") pod "a7ec94a9-a22f-4265-aef8-e7036a21cc3a" (UID: "a7ec94a9-a22f-4265-aef8-e7036a21cc3a"). InnerVolumeSpecName "kube-api-access-knsjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 00:58:35.051612 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:35.051581    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-openshift-service-ca\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:58:35.051612 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:35.051605    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knsjr\" (UniqueName: \"kubernetes.io/projected/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-kube-api-access-knsjr\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:58:35.051612 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:35.051615    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ec94a9-a22f-4265-aef8-e7036a21cc3a-maas-api-tls\") on node \"ip-10-0-128-116.ec2.internal\" DevicePath \"\""
Jul 10 00:58:35.191474 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:35.191437    2559 generic.go:356] "Generic (PLEG): container finished" podID="a7ec94a9-a22f-4265-aef8-e7036a21cc3a" containerID="d17d55de0f1b897ce8a6b0dc3c353d6036154d4fd5242e6c9446408ca2904de1" exitCode=0
Jul 10 00:58:35.191639 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:35.191484    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"
Jul 10 00:58:35.191639 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:35.191518    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk" event={"ID":"a7ec94a9-a22f-4265-aef8-e7036a21cc3a","Type":"ContainerDied","Data":"d17d55de0f1b897ce8a6b0dc3c353d6036154d4fd5242e6c9446408ca2904de1"}
Jul 10 00:58:35.191639 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:35.191573    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk" event={"ID":"a7ec94a9-a22f-4265-aef8-e7036a21cc3a","Type":"ContainerDied","Data":"9c3b7354b4abdaea99f104467583c56ebb41a50fecae642571c1b253baa8a1e1"}
Jul 10 00:58:35.191639 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:35.191595    2559 scope.go:117] "RemoveContainer" containerID="d17d55de0f1b897ce8a6b0dc3c353d6036154d4fd5242e6c9446408ca2904de1"
Jul 10 00:58:35.200603 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:35.200585    2559 scope.go:117] "RemoveContainer" containerID="d17d55de0f1b897ce8a6b0dc3c353d6036154d4fd5242e6c9446408ca2904de1"
Jul 10 00:58:35.200875 ip-10-0-128-116 kubenswrapper[2559]: E0710 00:58:35.200833    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d17d55de0f1b897ce8a6b0dc3c353d6036154d4fd5242e6c9446408ca2904de1\": container with ID starting with d17d55de0f1b897ce8a6b0dc3c353d6036154d4fd5242e6c9446408ca2904de1 not found: ID does not exist" containerID="d17d55de0f1b897ce8a6b0dc3c353d6036154d4fd5242e6c9446408ca2904de1"
Jul 10 00:58:35.200875 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:35.200858    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d17d55de0f1b897ce8a6b0dc3c353d6036154d4fd5242e6c9446408ca2904de1"} err="failed to get container status \"d17d55de0f1b897ce8a6b0dc3c353d6036154d4fd5242e6c9446408ca2904de1\": rpc error: code = NotFound desc = could not find container \"d17d55de0f1b897ce8a6b0dc3c353d6036154d4fd5242e6c9446408ca2904de1\": container with ID starting with d17d55de0f1b897ce8a6b0dc3c353d6036154d4fd5242e6c9446408ca2904de1 not found: ID does not exist"
Jul 10 00:58:35.213685 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:35.213657    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"]
Jul 10 00:58:35.217570 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:35.217540    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f84011-665f8fcf8-nxthk"]
Jul 10 00:58:36.836109 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:36.836079    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7ec94a9-a22f-4265-aef8-e7036a21cc3a" path="/var/lib/kubelet/pods/a7ec94a9-a22f-4265-aef8-e7036a21cc3a/volumes"
Jul 10 00:58:40.048091 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:40.048046    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:40.189951 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:40.189908    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:40.350075 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:40.349987    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:40.595782 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:40.595744    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:41.722363 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:41.722310    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:46.386600 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:46.386573    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-snjhz_3ed16b45-2f07-415b-be68-9d0da6cf6277/server/0.log"
Jul 10 00:58:46.902199 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:46.902174    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-559fc4999-zghf9_c55e31dc-0eac-449a-9d29-5805df85d1ce/copy-manifests/0.log"
Jul 10 00:58:46.908234 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:46.908204    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-559fc4999-zghf9_c55e31dc-0eac-449a-9d29-5805df85d1ce/manager/0.log"
Jul 10 00:58:47.259950 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:47.259920    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7bc864879c-27nt2_3673ec78-c8fc-4b38-a892-be81d2738ad2/maas-api/0.log"
Jul 10 00:58:49.335079 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:49.335047    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rrmln_405b6350-e566-49ab-81e0-59f70c86c6a3/manager/0.log"
Jul 10 00:58:50.080141 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:50.080106    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-sgkbf_ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25/discovery/0.log"
Jul 10 00:58:50.577768 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:50.577740    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-w2zng_f577c229-6648-4358-8cde-ab4bcf0a4ae9/payload-pre-processing/0.log"
Jul 10 00:58:51.308098 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:51.308014    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-4wkgc_e8a42e84-5cc0-4355-aeb4-a92a5f0cc41f/main/0.log"
Jul 10 00:58:51.425257 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:51.425228    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hv5t7_a84e1655-76f8-45b7-b2f2-ed59a51e244c/main/0.log"
Jul 10 00:58:55.538510 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:55.538475    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:56.643981 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:56.643948    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log" limit=1024
Jul 10 00:58:58.145084 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:58.145045    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6x94h_7ce01e87-f4a3-4774-b800-f1dc965f9689/global-pull-secret-syncer/0.log"
Jul 10 00:58:58.258295 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:58.258264    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cldmd_615763a8-ff81-4ac2-af5a-0b3c7b016222/konnectivity-agent/0.log"
Jul 10 00:58:58.355431 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:58:58.355396    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-116.ec2.internal_92d561d32137a202f7c75fa6c2ab0660/haproxy/0.log"
Jul 10 00:59:03.346192 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:03.346154    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rrmln_405b6350-e566-49ab-81e0-59f70c86c6a3/manager/0.log"
Jul 10 00:59:05.270139 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:05.270107    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-blndf_a2e6f08b-f22e-4ee7-b25d-d84eb08fb4f7/cluster-monitoring-operator/0.log"
Jul 10 00:59:05.375885 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:05.375856    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-r7btx_724ba4e9-fa99-42ac-afe2-b8044d78d131/monitoring-plugin/0.log"
Jul 10 00:59:05.552672 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:05.552641    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wgq7n_8faf0e88-2cb7-4551-b15c-bc1c31854b51/node-exporter/0.log"
Jul 10 00:59:05.574023 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:05.573994    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wgq7n_8faf0e88-2cb7-4551-b15c-bc1c31854b51/kube-rbac-proxy/0.log"
Jul 10 00:59:05.598802 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:05.598776    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wgq7n_8faf0e88-2cb7-4551-b15c-bc1c31854b51/init-textfile/0.log"
Jul 10 00:59:05.738090 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:05.738055    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cf3e8174-678c-472a-b5b6-5ca9235e10ef/prometheus/0.log"
Jul 10 00:59:05.772142 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:05.772120    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cf3e8174-678c-472a-b5b6-5ca9235e10ef/config-reloader/0.log"
Jul 10 00:59:05.805728 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:05.805706    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cf3e8174-678c-472a-b5b6-5ca9235e10ef/thanos-sidecar/0.log"
Jul 10 00:59:05.831733 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:05.831712    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cf3e8174-678c-472a-b5b6-5ca9235e10ef/kube-rbac-proxy-web/0.log"
Jul 10 00:59:05.851680 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:05.851657    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cf3e8174-678c-472a-b5b6-5ca9235e10ef/kube-rbac-proxy/0.log"
Jul 10 00:59:05.871110 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:05.871079    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cf3e8174-678c-472a-b5b6-5ca9235e10ef/kube-rbac-proxy-thanos/0.log"
Jul 10 00:59:05.888952 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:05.888933    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cf3e8174-678c-472a-b5b6-5ca9235e10ef/init-config-reloader/0.log"
Jul 10 00:59:05.914091 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:05.914071    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-vrgjp_4339eff7-c819-4551-94bc-468a6929c710/prometheus-operator/0.log"
Jul 10 00:59:05.934473 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:05.934448    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-vrgjp_4339eff7-c819-4551-94bc-468a6929c710/kube-rbac-proxy/0.log"
Jul 10 00:59:07.508501 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:07.508472    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-8j7st_44e54584-138b-4286-a17e-3e16fabe6861/networking-console-plugin/0.log"
Jul 10 00:59:08.774574 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:08.774544    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-cgdmm_0f28bafe-bd50-4228-aee1-04e545bbf94a/download-server/0.log"
Jul 10 00:59:09.400232 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.400198    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"]
Jul 10 00:59:09.400592 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.400579    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5" containerName="maas-api"
Jul 10 00:59:09.400647 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.400594    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5" containerName="maas-api"
Jul 10 00:59:09.400647 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.400607    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7ec94a9-a22f-4265-aef8-e7036a21cc3a" containerName="maas-api"
Jul 10 00:59:09.400647 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.400613    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7ec94a9-a22f-4265-aef8-e7036a21cc3a" containerName="maas-api"
Jul 10 00:59:09.400756 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.400665    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="cafc18d0-5b62-4c7d-bb06-7a9e9558ebd5" containerName="maas-api"
Jul 10 00:59:09.400756 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.400672    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7ec94a9-a22f-4265-aef8-e7036a21cc3a" containerName="maas-api"
Jul 10 00:59:09.403686 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.403667    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.406514 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.406496    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tcnxx\"/\"default-dockercfg-cv5kp\""
Jul 10 00:59:09.406616 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.406573    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tcnxx\"/\"kube-root-ca.crt\""
Jul 10 00:59:09.406685 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.406673    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tcnxx\"/\"openshift-service-ca.crt\""
Jul 10 00:59:09.411935 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.411911    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"]
Jul 10 00:59:09.448729 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.448700    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d7342d5d-cb4a-4b36-9263-e8efc1786252-podres\") pod \"perf-node-gather-daemonset-7fwz8\" (UID: \"d7342d5d-cb4a-4b36-9263-e8efc1786252\") " pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.448867 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.448772    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d7342d5d-cb4a-4b36-9263-e8efc1786252-proc\") pod \"perf-node-gather-daemonset-7fwz8\" (UID: \"d7342d5d-cb4a-4b36-9263-e8efc1786252\") " pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.448867 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.448810    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s74vw\" (UniqueName: \"kubernetes.io/projected/d7342d5d-cb4a-4b36-9263-e8efc1786252-kube-api-access-s74vw\") pod \"perf-node-gather-daemonset-7fwz8\" (UID: \"d7342d5d-cb4a-4b36-9263-e8efc1786252\") " pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.448867 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.448848    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d7342d5d-cb4a-4b36-9263-e8efc1786252-lib-modules\") pod \"perf-node-gather-daemonset-7fwz8\" (UID: \"d7342d5d-cb4a-4b36-9263-e8efc1786252\") " pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.448973 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.448880    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7342d5d-cb4a-4b36-9263-e8efc1786252-sys\") pod \"perf-node-gather-daemonset-7fwz8\" (UID: \"d7342d5d-cb4a-4b36-9263-e8efc1786252\") " pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.481376 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.481353    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-flrgf_e9477f6c-0ae3-4303-8207-30e23db3c4a3/volume-data-source-validator/0.log"
Jul 10 00:59:09.549973 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.549945    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d7342d5d-cb4a-4b36-9263-e8efc1786252-proc\") pod \"perf-node-gather-daemonset-7fwz8\" (UID: \"d7342d5d-cb4a-4b36-9263-e8efc1786252\") " pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.550133 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.549981    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s74vw\" (UniqueName: \"kubernetes.io/projected/d7342d5d-cb4a-4b36-9263-e8efc1786252-kube-api-access-s74vw\") pod \"perf-node-gather-daemonset-7fwz8\" (UID: \"d7342d5d-cb4a-4b36-9263-e8efc1786252\") " pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.550133 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.550010    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d7342d5d-cb4a-4b36-9263-e8efc1786252-lib-modules\") pod \"perf-node-gather-daemonset-7fwz8\" (UID: \"d7342d5d-cb4a-4b36-9263-e8efc1786252\") " pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.550133 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.550030    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7342d5d-cb4a-4b36-9263-e8efc1786252-sys\") pod \"perf-node-gather-daemonset-7fwz8\" (UID: \"d7342d5d-cb4a-4b36-9263-e8efc1786252\") " pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.550133 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.550074    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d7342d5d-cb4a-4b36-9263-e8efc1786252-podres\") pod \"perf-node-gather-daemonset-7fwz8\" (UID: \"d7342d5d-cb4a-4b36-9263-e8efc1786252\") " pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.550133 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.550095    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d7342d5d-cb4a-4b36-9263-e8efc1786252-proc\") pod \"perf-node-gather-daemonset-7fwz8\" (UID: \"d7342d5d-cb4a-4b36-9263-e8efc1786252\") " pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.550363 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.550169    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d7342d5d-cb4a-4b36-9263-e8efc1786252-lib-modules\") pod \"perf-node-gather-daemonset-7fwz8\" (UID: \"d7342d5d-cb4a-4b36-9263-e8efc1786252\") " pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.550363 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.550167    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7342d5d-cb4a-4b36-9263-e8efc1786252-sys\") pod \"perf-node-gather-daemonset-7fwz8\" (UID: \"d7342d5d-cb4a-4b36-9263-e8efc1786252\") " pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.550363 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.550172    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d7342d5d-cb4a-4b36-9263-e8efc1786252-podres\") pod \"perf-node-gather-daemonset-7fwz8\" (UID: \"d7342d5d-cb4a-4b36-9263-e8efc1786252\") " pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.558608 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.558584    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s74vw\" (UniqueName: \"kubernetes.io/projected/d7342d5d-cb4a-4b36-9263-e8efc1786252-kube-api-access-s74vw\") pod \"perf-node-gather-daemonset-7fwz8\" (UID: \"d7342d5d-cb4a-4b36-9263-e8efc1786252\") " pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.713687 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.713658    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:09.843188 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:09.843159    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"]
Jul 10 00:59:09.845995 ip-10-0-128-116 kubenswrapper[2559]: W0710 00:59:09.845965    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd7342d5d_cb4a_4b36_9263_e8efc1786252.slice/crio-1fdefa5a31b5b2965092da6ae78bd200a4bf4f7d5cdf4e74174865433b1fa797 WatchSource:0}: Error finding container 1fdefa5a31b5b2965092da6ae78bd200a4bf4f7d5cdf4e74174865433b1fa797: Status 404 returned error can't find the container with id 1fdefa5a31b5b2965092da6ae78bd200a4bf4f7d5cdf4e74174865433b1fa797
Jul 10 00:59:10.352914 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:10.352875    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8" event={"ID":"d7342d5d-cb4a-4b36-9263-e8efc1786252","Type":"ContainerStarted","Data":"c2769b84a854b1dde3d45714f53e3f05272d557a4e1a68e2c36a6f3b7f85a454"}
Jul 10 00:59:10.353084 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:10.352972    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8" event={"ID":"d7342d5d-cb4a-4b36-9263-e8efc1786252","Type":"ContainerStarted","Data":"1fdefa5a31b5b2965092da6ae78bd200a4bf4f7d5cdf4e74174865433b1fa797"}
Jul 10 00:59:10.353084 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:10.353004    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:10.368359 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:10.368290    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8" podStartSLOduration=1.3682757620000001 podStartE2EDuration="1.368275762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:59:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:59:10.366763519 +0000 UTC m=+2600.069548402" watchObservedRunningTime="2026-07-10 00:59:10.368275762 +0000 UTC m=+2600.071060624"
Jul 10 00:59:10.534202 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:10.534174    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j8x4j_6e7a4774-431a-40c0-8c44-f3f929be8e0f/dns/0.log"
Jul 10 00:59:10.555260 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:10.555226    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j8x4j_6e7a4774-431a-40c0-8c44-f3f929be8e0f/kube-rbac-proxy/0.log"
Jul 10 00:59:10.661864 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:10.661795    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wtmzf_bc636ec8-6508-407c-8f57-38cf0bcf86a3/dns-node-resolver/0.log"
Jul 10 00:59:11.244465 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:11.244428    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gphtp_7ba31ebb-c51d-4692-80de-d8a7384b7e2c/node-ca/0.log"
Jul 10 00:59:12.269832 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:12.269755    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-sgkbf_ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25/discovery/0.log"
Jul 10 00:59:12.417734 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:12.417702    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-w2zng_f577c229-6648-4358-8cde-ab4bcf0a4ae9/payload-pre-processing/0.log"
Jul 10 00:59:13.185502 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:13.185475    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vtm7t_1d2b0c9d-41f9-45be-a60c-52fa7cb19f15/serve-healthcheck-canary/0.log"
Jul 10 00:59:13.620164 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:13.620119    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-dnvt2_db79ea28-a194-4f03-ab3c-a6faee9a011d/insights-operator/0.log"
Jul 10 00:59:13.620652 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:13.620536    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-dnvt2_db79ea28-a194-4f03-ab3c-a6faee9a011d/insights-operator/1.log"
Jul 10 00:59:13.772118 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:13.772080    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zdkv9_ed80c3b6-d4f4-490e-a89e-874824a1a195/kube-rbac-proxy/0.log"
Jul 10 00:59:13.789773 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:13.789753    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zdkv9_ed80c3b6-d4f4-490e-a89e-874824a1a195/exporter/0.log"
Jul 10 00:59:13.807302 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:13.807281    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zdkv9_ed80c3b6-d4f4-490e-a89e-874824a1a195/extractor/0.log"
Jul 10 00:59:16.306834 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:16.306806    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-snjhz_3ed16b45-2f07-415b-be68-9d0da6cf6277/server/0.log"
Jul 10 00:59:16.366908 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:16.366879    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tcnxx/perf-node-gather-daemonset-7fwz8"
Jul 10 00:59:16.597047 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:16.596973    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-559fc4999-zghf9_c55e31dc-0eac-449a-9d29-5805df85d1ce/manager/0.log"
Jul 10 00:59:16.614618 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:16.614594    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-559fc4999-zghf9_c55e31dc-0eac-449a-9d29-5805df85d1ce/copy-manifests/0.log"
Jul 10 00:59:17.711838 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:17.711810    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7b4d8999df-lzz45_a3a76c30-d768-4393-935e-7eb2240fc615/manager/0.log"
Jul 10 00:59:19.566502 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:19.566409    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-sgkbf_ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25/discovery/0.log"
Jul 10 00:59:19.683722 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:19.683690    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-w2zng_f577c229-6648-4358-8cde-ab4bcf0a4ae9/payload-pre-processing/0.log"
Jul 10 00:59:21.070413 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:21.070327    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-sgkbf_ddb4a0ba-d7b1-4d98-acf5-dca9abb9ea25/discovery/0.log"
Jul 10 00:59:21.200269 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:21.200240    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-w2zng_f577c229-6648-4358-8cde-ab4bcf0a4ae9/payload-pre-processing/0.log"
Jul 10 00:59:23.521111 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:23.521080    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-tsm82_0eef5919-029b-4087-93e6-c44b8fdce6fa/kube-storage-version-migrator-operator/1.log"
Jul 10 00:59:23.522178 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:23.522153    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-tsm82_0eef5919-029b-4087-93e6-c44b8fdce6fa/kube-storage-version-migrator-operator/0.log"
Jul 10 00:59:24.439235 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:24.439197    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-529b2_1e2cf300-7c42-4aaf-97d0-e85b66d2c2b6/kube-multus/0.log"
Jul 10 00:59:24.778224 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:24.778195    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tdzvd_6e12398d-7b1b-478b-8031-8b0e7617091d/kube-multus-additional-cni-plugins/0.log"
Jul 10 00:59:24.794669 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:24.794645    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tdzvd_6e12398d-7b1b-478b-8031-8b0e7617091d/egress-router-binary-copy/0.log"
Jul 10 00:59:24.810837 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:24.810818    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tdzvd_6e12398d-7b1b-478b-8031-8b0e7617091d/cni-plugins/0.log"
Jul 10 00:59:24.826308 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:24.826288    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tdzvd_6e12398d-7b1b-478b-8031-8b0e7617091d/bond-cni-plugin/0.log"
Jul 10 00:59:24.842030 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:24.842014    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tdzvd_6e12398d-7b1b-478b-8031-8b0e7617091d/routeoverride-cni/0.log"
Jul 10 00:59:24.857947 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:24.857930    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tdzvd_6e12398d-7b1b-478b-8031-8b0e7617091d/whereabouts-cni-bincopy/0.log"
Jul 10 00:59:24.874008 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:24.873983    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tdzvd_6e12398d-7b1b-478b-8031-8b0e7617091d/whereabouts-cni/0.log"
Jul 10 00:59:24.894568 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:24.894541    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qzdmp_1535944a-df71-496e-955d-7d67d827bec0/network-metrics-daemon/0.log"
Jul 10 00:59:24.909014 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:24.908993    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qzdmp_1535944a-df71-496e-955d-7d67d827bec0/kube-rbac-proxy/0.log"
Jul 10 00:59:26.022088 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:26.022056    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-controller/0.log"
Jul 10 00:59:26.037370 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:26.037338    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/1.log"
Jul 10 00:59:26.037370 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:26.037354    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovn-acl-logging/0.log"
Jul 10 00:59:26.068582 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:26.068556    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/kube-rbac-proxy-node/0.log"
Jul 10 00:59:26.087562 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:26.087533    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/kube-rbac-proxy-ovn-metrics/0.log"
Jul 10 00:59:26.100834 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:26.100814    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/northd/0.log"
Jul 10 00:59:26.117271 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:26.117244    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/nbdb/0.log"
Jul 10 00:59:26.134526 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:26.134504    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/sbdb/0.log"
Jul 10 00:59:26.177199 ip-10-0-128-116 kubenswrapper[2559]: I0710 00:59:26.177164    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2s7q_f75ad69a-e4b3-484a-a93c-468bd16e460c/ovnkube-controller/0.log"