May 29 02:24:56.949709 ip-10-0-133-245 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
May 29 02:24:56.949826 ip-10-0-133-245 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
May 29 02:24:56.949888 ip-10-0-133-245 systemd[1]: kubelet.service: Failed with result 'resources'.
May 29 02:24:56.950196 ip-10-0-133-245 systemd[1]: Failed to start Kubernetes Kubelet.
May 29 02:25:08.081816 ip-10-0-133-245 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
May 29 02:25:08.081835 ip-10-0-133-245 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 5f1d3533183a47cc9fcdaecb1ab0964f --
May 29 02:27:39.021052 ip-10-0-133-245 systemd[1]: Starting Kubernetes Kubelet...
May 29 02:27:39.452026 ip-10-0-133-245 kubenswrapper[2553]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 29 02:27:39.452026 ip-10-0-133-245 kubenswrapper[2553]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 29 02:27:39.452026 ip-10-0-133-245 kubenswrapper[2553]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 29 02:27:39.452026 ip-10-0-133-245 kubenswrapper[2553]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 29 02:27:39.452026 ip-10-0-133-245 kubenswrapper[2553]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 29 02:27:39.453732 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.453626    2553 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 29 02:27:39.457409 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457394    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457410    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457414    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457418    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457420    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457424    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457426    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457429    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457432    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457434    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457437    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457439    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457442    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457445    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457447    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457450    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457453    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457456    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 02:27:39.457451 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457459    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457462    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457465    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457467    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457470    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457472    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457474    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457477    2553 feature_gate.go:328] unrecognized feature gate: Example
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457480    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457482    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457484    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457487    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457489    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457492    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457494    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457497    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457499    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457501    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457504    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457507    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 02:27:39.457886 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457509    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457512    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457515    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457517    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457520    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457522    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457525    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457527    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457529    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457532    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457534    2553 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457537    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457539    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457542    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457545    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457548    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457550    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457554    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457558    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 02:27:39.458428 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457561    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457565    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457569    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457572    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457575    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457579    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457581    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457584    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457586    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457589    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457592    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457594    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457597    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457599    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457602    2553 feature_gate.go:328] unrecognized feature gate: Example2
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457604    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457607    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457609    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457611    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457614    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 02:27:39.458903 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457616    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457619    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457621    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457624    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457626    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457629    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457631    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457635    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.457637    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458070    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458075    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458078    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458081    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458083    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458087    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458089    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458092    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458095    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458097    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 02:27:39.459375 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458099    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458102    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458104    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458107    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458109    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458112    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458115    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458117    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458120    2553 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458122    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458125    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458127    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458130    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458132    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458134    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458137    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458139    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458142    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458144    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 02:27:39.459841 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458147    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458150    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458152    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458155    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458157    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458160    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458162    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458164    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458167    2553 feature_gate.go:328] unrecognized feature gate: Example2
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458170    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458173    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458175    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458177    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458181    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458183    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458185    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458188    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458190    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458193    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458196    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 02:27:39.460294 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458198    2553 feature_gate.go:328] unrecognized feature gate: Example
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458201    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458203    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458206    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458208    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458210    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458213    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458216    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458218    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458220    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458222    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458225    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458227    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458230    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458233    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458235    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458237    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458240    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458242    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 02:27:39.460791 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458245    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458249    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458252    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458261    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458264    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458266    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458269    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458271    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458274    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458276    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458278    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458284    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458287    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458289    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458292    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458294    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458298    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.458301    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459135    2553 flags.go:64] FLAG: --address="0.0.0.0"
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459144    2553 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 29 02:27:39.461254 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459154    2553 flags.go:64] FLAG: --anonymous-auth="true"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459158    2553 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459163    2553 flags.go:64] FLAG: --authentication-token-webhook="false"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459166    2553 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459170    2553 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459174    2553 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459177    2553 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459180    2553 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459184    2553 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459187    2553 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459191    2553 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459194    2553 flags.go:64] FLAG: --cgroup-root=""
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459197    2553 flags.go:64] FLAG: --cgroups-per-qos="true"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459200    2553 flags.go:64] FLAG: --client-ca-file=""
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459202    2553 flags.go:64] FLAG: --cloud-config=""
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459205    2553 flags.go:64] FLAG: --cloud-provider="external"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459209    2553 flags.go:64] FLAG: --cluster-dns="[]"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459216    2553 flags.go:64] FLAG: --cluster-domain=""
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459219    2553 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459222    2553 flags.go:64] FLAG: --config-dir=""
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459225    2553 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459228    2553 flags.go:64] FLAG: --container-log-max-files="5"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459232    2553 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459235    2553 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 29 02:27:39.461745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459238    2553 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459242    2553 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459245    2553 flags.go:64] FLAG: --contention-profiling="false"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459248    2553 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459251    2553 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459254    2553 flags.go:64] FLAG: --cpu-manager-policy="none"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459257    2553 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459261    2553 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459264    2553 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459267    2553 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459271    2553 flags.go:64] FLAG: --enable-load-reader="false"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459274    2553 flags.go:64] FLAG: --enable-server="true"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459276    2553 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459281    2553 flags.go:64] FLAG: --event-burst="100"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459285    2553 flags.go:64] FLAG: --event-qps="50"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459287    2553 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459291    2553 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459294    2553 flags.go:64] FLAG: --eviction-hard=""
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459297    2553 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459300    2553 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459303    2553 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459306    2553 flags.go:64] FLAG: --eviction-soft=""
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459309    2553 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459312    2553 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459314    2553 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 29 02:27:39.462310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459318    2553 flags.go:64] FLAG: --experimental-mounter-path=""
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459321    2553 flags.go:64] FLAG: --fail-cgroupv1="false"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459324    2553 flags.go:64] FLAG: --fail-swap-on="true"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459326    2553 flags.go:64] FLAG: --feature-gates=""
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459330    2553 flags.go:64] FLAG: --file-check-frequency="20s"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459333    2553 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459337    2553 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459340    2553 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459343    2553 flags.go:64] FLAG: --healthz-port="10248"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459346    2553 flags.go:64] FLAG: --help="false"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459349    2553 flags.go:64] FLAG: --hostname-override="ip-10-0-133-245.ec2.internal"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459352    2553 flags.go:64] FLAG: --housekeeping-interval="10s"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459355    2553 flags.go:64] FLAG: --http-check-frequency="20s"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459358    2553 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459361    2553 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459365    2553 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459367    2553 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459370    2553 flags.go:64] FLAG: --image-service-endpoint=""
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459373    2553 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459376    2553 flags.go:64] FLAG: --kube-api-burst="100"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459379    2553 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459382    2553 flags.go:64] FLAG: --kube-api-qps="50"
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459386    2553 flags.go:64] FLAG: --kube-reserved=""
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459389    2553 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 29 02:27:39.462916 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459391    2553 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459394    2553 flags.go:64] FLAG: --kubelet-cgroups=""
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459397    2553 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459400    2553 flags.go:64] FLAG: --lock-file=""
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459402    2553 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459405    2553 flags.go:64] FLAG: --log-flush-frequency="5s"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459408    2553 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459413    2553 flags.go:64] FLAG: --log-json-split-stream="false"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459416    2553 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459419    2553 flags.go:64] FLAG: --log-text-split-stream="false"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459422    2553 flags.go:64] FLAG: --logging-format="text"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459425    2553 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459429    2553 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459432    2553 flags.go:64] FLAG: --manifest-url=""
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459437    2553 flags.go:64] FLAG: --manifest-url-header=""
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459442    2553 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459444    2553 flags.go:64] FLAG: --max-open-files="1000000"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459449    2553 flags.go:64] FLAG: --max-pods="110"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459452    2553 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459455    2553 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459458    2553 flags.go:64] FLAG: --memory-manager-policy="None"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459461    2553 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459464    2553 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459467    2553 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459470    2553 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 29 02:27:39.463526 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459477    2553 flags.go:64] FLAG: --node-status-max-images="50"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459480    2553 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459483    2553 flags.go:64] FLAG: --oom-score-adj="-999"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459486    2553 flags.go:64] FLAG: --pod-cidr=""
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459489    2553 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459494    2553 flags.go:64] FLAG: --pod-manifest-path=""
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459497    2553 flags.go:64] FLAG: --pod-max-pids="-1"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459500    2553 flags.go:64] FLAG: --pods-per-core="0"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459503    2553 flags.go:64] FLAG: --port="10250"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459506    2553 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459509    2553 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fe4569d1844a7dc9"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459512    2553 flags.go:64] FLAG: --qos-reserved=""
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459515    2553 flags.go:64] FLAG: --read-only-port="10255"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459518    2553 flags.go:64] FLAG: --register-node="true"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459521    2553 flags.go:64] FLAG: --register-schedulable="true"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459524    2553 flags.go:64] FLAG: --register-with-taints=""
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459527    2553 flags.go:64] FLAG: --registry-burst="10"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459530    2553 flags.go:64] FLAG: --registry-qps="5"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459534    2553 flags.go:64] FLAG: --reserved-cpus=""
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459536    2553 flags.go:64] FLAG: --reserved-memory=""
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459540    2553 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459545    2553 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459548    2553 flags.go:64] FLAG: --rotate-certificates="false"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459551    2553 flags.go:64] FLAG: --rotate-server-certificates="false"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459554    2553 flags.go:64] FLAG: --runonce="false"
May 29 02:27:39.464152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459557    2553 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459560    2553 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459563    2553 flags.go:64] FLAG: --seccomp-default="false"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459566    2553 flags.go:64] FLAG: --serialize-image-pulls="true"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459568    2553 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459571    2553 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459575    2553 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459578    2553 flags.go:64] FLAG: --storage-driver-password="root"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459581    2553 flags.go:64] FLAG: --storage-driver-secure="false"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459584    2553 flags.go:64] FLAG: --storage-driver-table="stats"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459587    2553 flags.go:64] FLAG: --storage-driver-user="root"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459590    2553 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459593    2553 flags.go:64] FLAG: --sync-frequency="1m0s"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459596    2553 flags.go:64] FLAG: --system-cgroups=""
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459598    2553 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459604    2553 flags.go:64] FLAG: --system-reserved-cgroup=""
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459606    2553 flags.go:64] FLAG: --tls-cert-file=""
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459609    2553 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459613    2553 flags.go:64] FLAG: --tls-min-version=""
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459615    2553 flags.go:64] FLAG: --tls-private-key-file=""
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459618    2553 flags.go:64] FLAG: --topology-manager-policy="none"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459621    2553 flags.go:64] FLAG: --topology-manager-policy-options=""
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459624    2553 flags.go:64] FLAG: --topology-manager-scope="container"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459627    2553 flags.go:64] FLAG: --v="2"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459631    2553 flags.go:64] FLAG: --version="false"
May 29 02:27:39.464762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459636    2553 flags.go:64] FLAG: --vmodule=""
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459640    2553 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.459643    2553 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459751    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459755    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459760    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459765    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459770    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459774    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459779    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459784    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459787    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459790    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459792    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459795    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459798    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459800    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459802    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459805    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459808    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 02:27:39.465403 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459811    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459813    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459816    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459819    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459821    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459823    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459826    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459829    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459831    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459834    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459836    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459840    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459843    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459848    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459853    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459858    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459862    2553 feature_gate.go:328] unrecognized feature gate: Example
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459866    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459869    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459872    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 02:27:39.465897 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459874    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459877    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459880    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459882    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459884    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459887    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459889    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459892    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459894    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459897    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459900    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459906    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459909    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459912    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459914    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459917    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459919    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459922    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459924    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459927    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 02:27:39.466418 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459931    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459935    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459939    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459944    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459949    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459953    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459957    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459960    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459963    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459965    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459968    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459971    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459973    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459975    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459978    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459981    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459983    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459986    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459988    2553 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 02:27:39.466949 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459991    2553 feature_gate.go:328] unrecognized feature gate: Example2
May 29 02:27:39.467696 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459993    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 02:27:39.467696 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459996    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 02:27:39.467696 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.459998    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 02:27:39.467696 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.460002    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 02:27:39.467696 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.460005    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 02:27:39.467696 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.460007    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 02:27:39.467696 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.460010    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 02:27:39.467696 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.460015    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 02:27:39.467696 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.460019    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 02:27:39.467696 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.460738    2553 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 29 02:27:39.469956 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.469829    2553 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 29 02:27:39.469956 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.469953    2553 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470022    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470030    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470035    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470039    2553 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470044    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470049    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470053    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470057    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470061    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470066    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470070    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470074    2553 feature_gate.go:328] unrecognized feature gate: Example
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470078    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470083    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470087    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470092    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470096    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470100    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470105    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 02:27:39.470120 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470109    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470113    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470117    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470121    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470125    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470129    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470134    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470138    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470142    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470146    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470152    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470157    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470161    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470165    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470169    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470173    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470178    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470182    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470186    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470190    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 02:27:39.471003 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470194    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470199    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470203    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470206    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470211    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470215    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470219    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470223    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470227    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470231    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470235    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470240    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470244    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470250    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470257    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470262    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470266    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470271    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470275    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 02:27:39.471550 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470279    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470283    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470287    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470291    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470296    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470300    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470304    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470308    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470312    2553 feature_gate.go:328] unrecognized feature gate: Example2
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470316    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470320    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470325    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470330    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470334    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470339    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470343    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470347    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470351    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470357    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470362    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 02:27:39.472116 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470367    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 02:27:39.472826 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470371    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 02:27:39.472826 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470375    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 02:27:39.472826 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470379    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 02:27:39.472826 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470383    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 02:27:39.472826 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470390    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 02:27:39.472826 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470397    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 02:27:39.472826 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470403    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 02:27:39.472826 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.470412    2553 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 29 02:27:39.472826 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470567    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 02:27:39.472826 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470576    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 02:27:39.472826 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470580    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 02:27:39.472826 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470585    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 02:27:39.472826 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470590    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 02:27:39.472826 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470594    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 02:27:39.472826 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470598    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470602    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470606    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470610    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470615    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470619    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470623    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470627    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470631    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470636    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470640    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470644    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470648    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470671    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470675    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470679    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470686    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470691    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470695    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470701    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 02:27:39.473291 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470708    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470712    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470716    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470720    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470724    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470728    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470732    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470736    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470740    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470744    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470749    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470753    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470757    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470761    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470765    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470769    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470773    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470778    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470784    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 02:27:39.473803 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470789    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470793    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470798    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470802    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470806    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470811    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470816    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470820    2553 feature_gate.go:328] unrecognized feature gate: Example
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470824    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470829    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470834    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470838    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470842    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470846    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470850    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470854    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470858    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470862    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470866    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470870    2553 feature_gate.go:328] unrecognized feature gate: Example2
May 29 02:27:39.474283 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470874    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470878    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470882    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470886    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470890    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470894    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470898    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470902    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470906    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470910    2553 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470914    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470918    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470921    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470925    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470929    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470933    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470937    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470941    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470945    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470949    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 02:27:39.474786 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:39.470953    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 02:27:39.475594 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.470960    2553 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 29 02:27:39.475594 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.471617    2553 server.go:962] "Client rotation is on, will bootstrap in background"
May 29 02:27:39.476483 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.476466    2553 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 29 02:27:39.477476 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.477463    2553 server.go:1019] "Starting client certificate rotation"
May 29 02:27:39.477587 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.477569    2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 29 02:27:39.477621 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.477614    2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 29 02:27:39.500644 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.500586    2553 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 29 02:27:39.503252 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.503230    2553 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 29 02:27:39.521730 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.521714    2553 log.go:25] "Validated CRI v1 runtime API"
May 29 02:27:39.529927 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.529897    2553 log.go:25] "Validated CRI v1 image API"
May 29 02:27:39.531421 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.531398    2553 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 29 02:27:39.537950 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.537922    2553 fs.go:135] Filesystem UUIDs: map[5a8e821c-a51b-4719-ba8b-2ff7a1f303e4:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 bf937ed9-231b-402d-a02d-930e3a62da09:/dev/nvme0n1p4]
May 29 02:27:39.538019 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.537949    2553 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 29 02:27:39.538362 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.538345    2553 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 29 02:27:39.542389 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.542283    2553 manager.go:217] Machine: {Timestamp:2026-05-29 02:27:39.541805362 +0000 UTC m=+0.405417884 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099357 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22dac6d3314e27fdab4d8a49afd17e SystemUUID:ec22dac6-d331-4e27-fdab-4d8a49afd17e BootID:5f1d3533-183a-47cc-9fcd-aecb1ab0964f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:10:c4:ae:7c:3d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:10:c4:ae:7c:3d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:2d:7a:30:61:16 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 29 02:27:39.543147 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.543135    2553 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 29 02:27:39.543265 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.543253    2553 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 29 02:27:39.545154 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.545128    2553 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 29 02:27:39.545325 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.545156    2553 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-245.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 29 02:27:39.545406 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.545340    2553 topology_manager.go:138] "Creating topology manager with none policy"
May 29 02:27:39.545406 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.545353    2553 container_manager_linux.go:306] "Creating device plugin manager"
May 29 02:27:39.545406 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.545364    2553 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 29 02:27:39.546175 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.546163    2553 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 29 02:27:39.547819 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.547806    2553 state_mem.go:36] "Initialized new in-memory state store"
May 29 02:27:39.547952 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.547941    2553 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 29 02:27:39.550289 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.550277    2553 kubelet.go:491] "Attempting to sync node with API server"
May 29 02:27:39.550345 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.550317    2553 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 29 02:27:39.550345 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.550334    2553 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 29 02:27:39.550438 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.550379    2553 kubelet.go:397] "Adding apiserver pod source"
May 29 02:27:39.550438 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.550392    2553 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 29 02:27:39.551550 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.551537    2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 29 02:27:39.551627 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.551560    2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 29 02:27:39.554592 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.554577    2553 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 29 02:27:39.561541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.561527    2553 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 29 02:27:39.563347 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.563335    2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 29 02:27:39.563400 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.563353    2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 29 02:27:39.563400 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.563359    2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 29 02:27:39.563400 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.563365    2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 29 02:27:39.563400 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.563373    2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 29 02:27:39.563400 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.563378    2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 29 02:27:39.563400 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.563384    2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 29 02:27:39.563400 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.563389    2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 29 02:27:39.563400 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.563396    2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 29 02:27:39.563400 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.563402    2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 29 02:27:39.563625 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.563410    2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 29 02:27:39.563625 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.563415    2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 29 02:27:39.564197 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.564186    2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 29 02:27:39.564197 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.564195    2553 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 29 02:27:39.566763 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.566745    2553 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-245.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 29 02:27:39.566999 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:39.566975    2553 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-245.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 29 02:27:39.567076 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:39.567009    2553 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 29 02:27:39.567231 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.567217    2553 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 29 02:27:39.567288 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.567269    2553 server.go:1295] "Started kubelet"
May 29 02:27:39.567373 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.567342    2553 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 29 02:27:39.567821 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.567794    2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cl9qh"
May 29 02:27:39.567903 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.567807    2553 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 29 02:27:39.567903 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.567873    2553 server_v1.go:47] "podresources" method="list" useActivePods=true
May 29 02:27:39.568265 ip-10-0-133-245 systemd[1]: Started Kubernetes Kubelet.
May 29 02:27:39.569247 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.569218    2553 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 29 02:27:39.569822 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.569809    2553 server.go:317] "Adding debug handlers to kubelet server"
May 29 02:27:39.576924 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.576892    2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cl9qh"
May 29 02:27:39.576924 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.576921    2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 29 02:27:39.577672 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.577635    2553 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 29 02:27:39.577988 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:39.577930    2553 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 29 02:27:39.577988 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.577960    2553 volume_manager.go:295] "The desired_state_of_world populator starts"
May 29 02:27:39.577988 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.577972    2553 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 29 02:27:39.578142 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.578108    2553 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 29 02:27:39.578272 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.578249    2553 reconstruct.go:97] "Volume reconstruction finished"
May 29 02:27:39.578272 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.578263    2553 reconciler.go:26] "Reconciler: start to sync state"
May 29 02:27:39.580935 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.579932    2553 factory.go:153] Registering CRI-O factory
May 29 02:27:39.580935 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.579950    2553 factory.go:223] Registration of the crio container factory successfully
May 29 02:27:39.580935 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.580000    2553 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 29 02:27:39.580935 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.580387    2553 factory.go:55] Registering systemd factory
May 29 02:27:39.580935 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.580393    2553 factory.go:223] Registration of the systemd container factory successfully
May 29 02:27:39.580935 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.580422    2553 factory.go:103] Registering Raw factory
May 29 02:27:39.580935 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.580444    2553 manager.go:1196] Started watching for new ooms in manager
May 29 02:27:39.581515 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.581499    2553 manager.go:319] Starting recovery of all containers
May 29 02:27:39.581619 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:39.581601    2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:39.587928 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.587908    2553 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 02:27:39.590338 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:39.590312    2553 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-245.ec2.internal\" not found" node="ip-10-0-133-245.ec2.internal"
May 29 02:27:39.591600 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.591582    2553 manager.go:324] Recovery completed
May 29 02:27:39.595647 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.595634    2553 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 02:27:39.597505 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.597489    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasSufficientMemory"
May 29 02:27:39.597568 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.597520    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasNoDiskPressure"
May 29 02:27:39.597568 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.597534    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasSufficientPID"
May 29 02:27:39.597999 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.597987    2553 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 29 02:27:39.597999 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.597998    2553 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 29 02:27:39.598097 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.598013    2553 state_mem.go:36] "Initialized new in-memory state store"
May 29 02:27:39.601040 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.601028    2553 policy_none.go:49] "None policy: Start"
May 29 02:27:39.601091 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.601044    2553 memory_manager.go:186] "Starting memorymanager" policy="None"
May 29 02:27:39.601091 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.601053    2553 state_mem.go:35] "Initializing new in-memory state store"
May 29 02:27:39.636224 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.636209    2553 manager.go:341] "Starting Device Plugin manager"
May 29 02:27:39.651907 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:39.636243    2553 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 29 02:27:39.651907 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.636261    2553 server.go:85] "Starting device plugin registration server"
May 29 02:27:39.651907 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.636474    2553 eviction_manager.go:189] "Eviction manager: starting control loop"
May 29 02:27:39.651907 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.636483    2553 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 29 02:27:39.651907 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.636599    2553 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 29 02:27:39.651907 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.636734    2553 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 29 02:27:39.651907 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.636746    2553 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 29 02:27:39.651907 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:39.637585    2553 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 29 02:27:39.651907 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:39.637615    2553 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:39.714034 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.714004    2553 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 29 02:27:39.715327 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.715310    2553 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 29 02:27:39.715387 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.715337    2553 status_manager.go:230] "Starting to sync pod status with apiserver"
May 29 02:27:39.715387 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.715362    2553 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 29 02:27:39.715387 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.715371    2553 kubelet.go:2451] "Starting kubelet main sync loop"
May 29 02:27:39.715483 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:39.715465    2553 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 29 02:27:39.718813 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.718791    2553 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 02:27:39.736699 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.736679    2553 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 02:27:39.737451 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.737434    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasSufficientMemory"
May 29 02:27:39.737509 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.737464    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasNoDiskPressure"
May 29 02:27:39.737509 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.737474    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasSufficientPID"
May 29 02:27:39.737509 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.737494    2553 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-245.ec2.internal"
May 29 02:27:39.747069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.747050    2553 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-245.ec2.internal"
May 29 02:27:39.747128 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:39.747069    2553 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-245.ec2.internal\": node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:39.769114 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:39.769097    2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:39.816378 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.816354    2553 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-245.ec2.internal"]
May 29 02:27:39.816444 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.816436    2553 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 02:27:39.817220 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.817205    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasSufficientMemory"
May 29 02:27:39.817294 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.817236    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasNoDiskPressure"
May 29 02:27:39.817294 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.817245    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasSufficientPID"
May 29 02:27:39.818849 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.818837    2553 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 02:27:39.818999 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.818985    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal"
May 29 02:27:39.819044 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.819011    2553 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 02:27:39.819505 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.819490    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasSufficientMemory"
May 29 02:27:39.819578 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.819517    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasNoDiskPressure"
May 29 02:27:39.819578 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.819530    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasSufficientPID"
May 29 02:27:39.819578 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.819545    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasSufficientMemory"
May 29 02:27:39.819578 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.819563    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasNoDiskPressure"
May 29 02:27:39.819578 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.819572    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasSufficientPID"
May 29 02:27:39.822265 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.821125    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-245.ec2.internal"
May 29 02:27:39.822265 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.821193    2553 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 02:27:39.822635 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.822614    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasSufficientMemory"
May 29 02:27:39.822752 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.822646    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasNoDiskPressure"
May 29 02:27:39.822752 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.822682    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeHasSufficientPID"
May 29 02:27:39.847211 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:39.847189    2553 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-245.ec2.internal\" not found" node="ip-10-0-133-245.ec2.internal"
May 29 02:27:39.851521 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:39.851506    2553 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-245.ec2.internal\" not found" node="ip-10-0-133-245.ec2.internal"
May 29 02:27:39.869539 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:39.869522    2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:39.879550 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.879530    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bf5e5a7948545e5d9a23a951bc43bf6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal\" (UID: \"5bf5e5a7948545e5d9a23a951bc43bf6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal"
May 29 02:27:39.879616 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.879556    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f2cb1de6875396d67d2b36a572294b25-config\") pod \"kube-apiserver-proxy-ip-10-0-133-245.ec2.internal\" (UID: \"f2cb1de6875396d67d2b36a572294b25\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-245.ec2.internal"
May 29 02:27:39.879616 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.879573    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5bf5e5a7948545e5d9a23a951bc43bf6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal\" (UID: \"5bf5e5a7948545e5d9a23a951bc43bf6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal"
May 29 02:27:39.969590 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:39.969565    2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:39.979850 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.979831    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f2cb1de6875396d67d2b36a572294b25-config\") pod \"kube-apiserver-proxy-ip-10-0-133-245.ec2.internal\" (UID: \"f2cb1de6875396d67d2b36a572294b25\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-245.ec2.internal"
May 29 02:27:39.979904 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.979857    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5bf5e5a7948545e5d9a23a951bc43bf6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal\" (UID: \"5bf5e5a7948545e5d9a23a951bc43bf6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal"
May 29 02:27:39.979904 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.979878    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bf5e5a7948545e5d9a23a951bc43bf6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal\" (UID: \"5bf5e5a7948545e5d9a23a951bc43bf6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal"
May 29 02:27:39.979971 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.979927    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bf5e5a7948545e5d9a23a951bc43bf6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal\" (UID: \"5bf5e5a7948545e5d9a23a951bc43bf6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal"
May 29 02:27:39.979971 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.979938    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5bf5e5a7948545e5d9a23a951bc43bf6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal\" (UID: \"5bf5e5a7948545e5d9a23a951bc43bf6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal"
May 29 02:27:39.979971 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:39.979927    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f2cb1de6875396d67d2b36a572294b25-config\") pod \"kube-apiserver-proxy-ip-10-0-133-245.ec2.internal\" (UID: \"f2cb1de6875396d67d2b36a572294b25\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-245.ec2.internal"
May 29 02:27:40.070400 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:40.070341    2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:40.150824 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:40.150797    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal"
May 29 02:27:40.154806 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:40.154790    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-245.ec2.internal"
May 29 02:27:40.171291 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:40.171265    2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:40.271834 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:40.271797    2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:40.372318 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:40.372256    2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:40.472852 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:40.472815    2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:40.476961 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:40.476940    2553 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 29 02:27:40.477123 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:40.477088    2553 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 29 02:27:40.477123 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:40.477100    2553 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 29 02:27:40.573405 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:40.573375    2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:40.578432 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:40.578407    2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 29 02:27:40.580585 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:40.580558    2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-28 02:22:39 +0000 UTC" deadline="2027-12-07 10:57:12.445997851 +0000 UTC"
May 29 02:27:40.580585 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:40.580584    2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13376h29m31.865417358s"
May 29 02:27:40.601641 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:40.601493    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf5e5a7948545e5d9a23a951bc43bf6.slice/crio-6a5c38396b990df479547e88aec7611d0399c73a105e40b6d03153dec9f34506 WatchSource:0}: Error finding container 6a5c38396b990df479547e88aec7611d0399c73a105e40b6d03153dec9f34506: Status 404 returned error can't find the container with id 6a5c38396b990df479547e88aec7611d0399c73a105e40b6d03153dec9f34506
May 29 02:27:40.602020 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:40.602002    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2cb1de6875396d67d2b36a572294b25.slice/crio-92dceca8c5d68f11eff28bcabe2109cca68872a263f863cf8d3c8faa84dcc6e9 WatchSource:0}: Error finding container 92dceca8c5d68f11eff28bcabe2109cca68872a263f863cf8d3c8faa84dcc6e9: Status 404 returned error can't find the container with id 92dceca8c5d68f11eff28bcabe2109cca68872a263f863cf8d3c8faa84dcc6e9
May 29 02:27:40.605600 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:40.605576    2553 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 29 02:27:40.606377 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:40.606355    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 02:27:40.626745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:40.626699    2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ddw26"
May 29 02:27:40.642671 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:40.642628    2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ddw26"
May 29 02:27:40.674453 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:40.674432    2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:40.718699 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:40.718641    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-245.ec2.internal" event={"ID":"f2cb1de6875396d67d2b36a572294b25","Type":"ContainerStarted","Data":"92dceca8c5d68f11eff28bcabe2109cca68872a263f863cf8d3c8faa84dcc6e9"}
May 29 02:27:40.719531 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:40.719482    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal" event={"ID":"5bf5e5a7948545e5d9a23a951bc43bf6","Type":"ContainerStarted","Data":"6a5c38396b990df479547e88aec7611d0399c73a105e40b6d03153dec9f34506"}
May 29 02:27:40.774698 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:40.774673    2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:40.875203 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:40.875157    2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:40.975845 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:40.975732    2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:41.076596 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:41.076561    2553 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-245.ec2.internal\" not found"
May 29 02:27:41.095640 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.095608    2553 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 02:27:41.166397 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.166368    2553 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 02:27:41.178593 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.178554    2553 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal"
May 29 02:27:41.193343 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.193319    2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 29 02:27:41.194454 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.194422    2553 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-245.ec2.internal"
May 29 02:27:41.204410 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.204391    2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 29 02:27:41.551921 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.551891    2553 apiserver.go:52] "Watching apiserver"
May 29 02:27:41.559244 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.559221    2553 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 29 02:27:41.561972 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.561945    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-245.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal","openshift-multus/multus-additional-cni-plugins-h6mc4","openshift-multus/network-metrics-daemon-7sv8v","openshift-network-operator/iptables-alerter-bx6zg","openshift-ovn-kubernetes/ovnkube-node-2mq8b","kube-system/konnectivity-agent-wvm6p","openshift-cluster-node-tuning-operator/tuned-qqvl8","openshift-dns/node-resolver-w7gt7","openshift-image-registry/node-ca-t8ct4","openshift-multus/multus-t8gs7","openshift-network-diagnostics/network-check-target-w5mh6"]
May 29 02:27:41.563387 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.563365    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.564555 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.564531    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.565778 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.565758    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.566282 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.566265    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 29 02:27:41.566438 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.566420    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 29 02:27:41.566548 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.566527    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 29 02:27:41.567367 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.566988    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-h4f9k\""
May 29 02:27:41.567576 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.567554    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 29 02:27:41.567794 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.567776    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 29 02:27:41.567875 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.567797    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 29 02:27:41.568337 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.568314    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 29 02:27:41.568476 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.568458    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 29 02:27:41.568771 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.568755    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5rz5l\""
May 29 02:27:41.569096 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.569083    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:41.569199 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:41.569179    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:27:41.571051 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.570525    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 29 02:27:41.571051 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.570826    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 29 02:27:41.571051 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.570832    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 29 02:27:41.571051 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.570746    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7vphr\""
May 29 02:27:41.571286 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.571088    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 29 02:27:41.571286 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.571190    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bx6zg"
May 29 02:27:41.571286 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.571249    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 29 02:27:41.571286 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.571192    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 29 02:27:41.574362 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.574306    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 29 02:27:41.574362 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.574337    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 29 02:27:41.574488 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.574379    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 29 02:27:41.574488 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.574467    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5b4wg\""
May 29 02:27:41.574620 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.574604    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wvm6p"
May 29 02:27:41.575934 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.575912    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.576743 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.576727    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 29 02:27:41.576938 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.576919    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-45trd\""
May 29 02:27:41.577126 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.577110    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w7gt7"
May 29 02:27:41.577216 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.577198    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 29 02:27:41.578068 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.578051    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sp5ch\""
May 29 02:27:41.578153 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.578086    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 29 02:27:41.578343 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.578325    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 29 02:27:41.578487 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.578470    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t8ct4"
May 29 02:27:41.579127 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.579110    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 29 02:27:41.579127 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.579123    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6qt46\""
May 29 02:27:41.579457 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.579439    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 29 02:27:41.580220 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.580202    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.580857 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.580835    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 29 02:27:41.580954 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.580859    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 29 02:27:41.580954 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.580890    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-45mz6\""
May 29 02:27:41.581075 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.580835    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 29 02:27:41.581516 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.581498    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:41.581583 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:41.581564    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:27:41.582377 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.582356    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-52m6z\""
May 29 02:27:41.582472 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.582461    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 29 02:27:41.589685 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.589666    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.589770 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.589694    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96b44fa7-1fa7-4811-8513-8d367a40ed40-host\") pod \"node-ca-t8ct4\" (UID: \"96b44fa7-1fa7-4811-8513-8d367a40ed40\") " pod="openshift-image-registry/node-ca-t8ct4"
May 29 02:27:41.589770 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.589714    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/870dbc4e-8a43-4aa8-babb-64b1aa538fdd-host-slash\") pod \"iptables-alerter-bx6zg\" (UID: \"870dbc4e-8a43-4aa8-babb-64b1aa538fdd\") " pod="openshift-network-operator/iptables-alerter-bx6zg"
May 29 02:27:41.589840 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.589769    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-cni-bin\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.589840 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.589803    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs\") pod \"network-metrics-daemon-7sv8v\" (UID: \"9a900d97-0943-4380-a1e5-85bd64a30ba3\") " pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:41.589840 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.589830    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-systemd\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.589949 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.589870    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-socket-dir\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.589949 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.589887    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1-hosts-file\") pod \"node-resolver-w7gt7\" (UID: \"d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1\") " pod="openshift-dns/node-resolver-w7gt7"
May 29 02:27:41.589949 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.589919    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-cnibin\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.590057 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.589950    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-device-dir\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.590057 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.589976    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-cni-binary-copy\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.590057 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590000    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-host\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.590057 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590027    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.590057 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590050    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/96b44fa7-1fa7-4811-8513-8d367a40ed40-serviceca\") pod \"node-ca-t8ct4\" (UID: \"96b44fa7-1fa7-4811-8513-8d367a40ed40\") " pod="openshift-image-registry/node-ca-t8ct4"
May 29 02:27:41.590229 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590072    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-run-openvswitch\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.590229 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590096    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5gp2\" (UniqueName: \"kubernetes.io/projected/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-kube-api-access-k5gp2\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.590229 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590127    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-sys\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.590229 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590152    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndw78\" (UniqueName: \"kubernetes.io/projected/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-kube-api-access-ndw78\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.590229 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590167    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.590229 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590199    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/870dbc4e-8a43-4aa8-babb-64b1aa538fdd-iptables-alerter-script\") pod \"iptables-alerter-bx6zg\" (UID: \"870dbc4e-8a43-4aa8-babb-64b1aa538fdd\") " pod="openshift-network-operator/iptables-alerter-bx6zg"
May 29 02:27:41.590229 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590215    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a1da5c79-fc3a-4c1e-b30b-44de60231e7e-konnectivity-ca\") pod \"konnectivity-agent-wvm6p\" (UID: \"a1da5c79-fc3a-4c1e-b30b-44de60231e7e\") " pod="kube-system/konnectivity-agent-wvm6p"
May 29 02:27:41.590229 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590228    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-log-socket\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.590498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590246    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-cni-netd\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.590498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590270    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-ovnkube-script-lib\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.590498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590288    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1-tmp-dir\") pod \"node-resolver-w7gt7\" (UID: \"d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1\") " pod="openshift-dns/node-resolver-w7gt7"
May 29 02:27:41.590498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590302    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-system-cni-dir\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.590498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590315    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.590498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590344    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zf7f\" (UniqueName: \"kubernetes.io/projected/9a900d97-0943-4380-a1e5-85bd64a30ba3-kube-api-access-2zf7f\") pod \"network-metrics-daemon-7sv8v\" (UID: \"9a900d97-0943-4380-a1e5-85bd64a30ba3\") " pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:41.590498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590362    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-sysctl-conf\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.590498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590375    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-run-ovn\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.590498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590399    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-etc-selinux\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.590498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590433    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-var-lib-kubelet\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.590498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590455    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c565637d-58a5-405b-8e81-bf2131b39c29-etc-tuned\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.590498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590472    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-kubelet\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.591118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590521    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-run-systemd\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.591118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590555    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-slash\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.591118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590580    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-var-lib-openvswitch\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.591118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590608    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.591118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590675    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-registration-dir\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.591118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590699    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a1da5c79-fc3a-4c1e-b30b-44de60231e7e-agent-certs\") pod \"konnectivity-agent-wvm6p\" (UID: \"a1da5c79-fc3a-4c1e-b30b-44de60231e7e\") " pod="kube-system/konnectivity-agent-wvm6p"
May 29 02:27:41.591118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590723    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-sysconfig\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.591118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590765    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-kubernetes\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.591118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590787    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6clm8\" (UniqueName: \"kubernetes.io/projected/c565637d-58a5-405b-8e81-bf2131b39c29-kube-api-access-6clm8\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.591118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590822    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-ovnkube-config\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.591118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590846    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-env-overrides\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.591118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590869    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-modprobe-d\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.591118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590891    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-systemd-units\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.591118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590915    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-etc-openvswitch\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.591118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590938    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-sysctl-d\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.591118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590960    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-run\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.591590 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.590982    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c565637d-58a5-405b-8e81-bf2131b39c29-tmp\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.591590 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.591007    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-lib-modules\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.591590 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.591029    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-run-netns\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.591590 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.591052    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-run-ovn-kubernetes\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.591590 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.591099    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-ovn-node-metrics-cert\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.591590 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.591129    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcd56\" (UniqueName: \"kubernetes.io/projected/d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1-kube-api-access-gcd56\") pod \"node-resolver-w7gt7\" (UID: \"d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1\") " pod="openshift-dns/node-resolver-w7gt7"
May 29 02:27:41.591590 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.591146    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-os-release\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.591590 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.591169    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqqxf\" (UniqueName: \"kubernetes.io/projected/870dbc4e-8a43-4aa8-babb-64b1aa538fdd-kube-api-access-zqqxf\") pod \"iptables-alerter-bx6zg\" (UID: \"870dbc4e-8a43-4aa8-babb-64b1aa538fdd\") " pod="openshift-network-operator/iptables-alerter-bx6zg"
May 29 02:27:41.591590 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.591216    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzjl5\" (UniqueName: \"kubernetes.io/projected/96b44fa7-1fa7-4811-8513-8d367a40ed40-kube-api-access-bzjl5\") pod \"node-ca-t8ct4\" (UID: \"96b44fa7-1fa7-4811-8513-8d367a40ed40\") " pod="openshift-image-registry/node-ca-t8ct4"
May 29 02:27:41.591590 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.591241    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-node-log\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.591590 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.591263    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-sys-fs\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.591590 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.591284    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7664q\" (UniqueName: \"kubernetes.io/projected/ec97f75e-56a4-47d5-831f-e74157a2e7e2-kube-api-access-7664q\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.643679 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.643630    2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-28 02:22:40 +0000 UTC" deadline="2027-12-03 04:21:29.714517933 +0000 UTC"
May 29 02:27:41.643679 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.643677    2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13273h53m48.070843901s"
May 29 02:27:41.668697 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.668678    2553 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 02:27:41.679313 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.679279    2553 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 29 02:27:41.692406 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692384    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/870dbc4e-8a43-4aa8-babb-64b1aa538fdd-host-slash\") pod \"iptables-alerter-bx6zg\" (UID: \"870dbc4e-8a43-4aa8-babb-64b1aa538fdd\") " pod="openshift-network-operator/iptables-alerter-bx6zg"
May 29 02:27:41.692530 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692425    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-var-lib-cni-bin\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.692530 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692455    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-cni-bin\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.692530 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692481    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs\") pod \"network-metrics-daemon-7sv8v\" (UID: \"9a900d97-0943-4380-a1e5-85bd64a30ba3\") " pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:41.692530 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692505    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-systemd\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.692530 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692505    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/870dbc4e-8a43-4aa8-babb-64b1aa538fdd-host-slash\") pod \"iptables-alerter-bx6zg\" (UID: \"870dbc4e-8a43-4aa8-babb-64b1aa538fdd\") " pod="openshift-network-operator/iptables-alerter-bx6zg"
May 29 02:27:41.692768 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692515    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-cni-bin\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.692768 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692539    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-system-cni-dir\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.692768 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692565    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-hostroot\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.692768 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692589    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-systemd\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.692768 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692592    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-socket-dir\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.692768 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692632    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1-hosts-file\") pod \"node-resolver-w7gt7\" (UID: \"d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1\") " pod="openshift-dns/node-resolver-w7gt7"
May 29 02:27:41.692768 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:41.692644    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:27:41.692768 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692681    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-cnibin\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.692768 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692707    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-device-dir\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.692768 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692714    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-socket-dir\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.692768 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692729    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1-hosts-file\") pod \"node-resolver-w7gt7\" (UID: \"d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1\") " pod="openshift-dns/node-resolver-w7gt7"
May 29 02:27:41.692768 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:41.692740    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs podName:9a900d97-0943-4380-a1e5-85bd64a30ba3 nodeName:}" failed. No retries permitted until 2026-05-29 02:27:42.192709122 +0000 UTC m=+3.056321636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs") pod "network-metrics-daemon-7sv8v" (UID: "9a900d97-0943-4380-a1e5-85bd64a30ba3") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:27:41.692768 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692749    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-device-dir\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.692768 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692756    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-cnibin\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.692768 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692772    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-cni-binary-copy\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692794    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-host\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692811    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-cni-binary-copy\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692832    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k6rr\" (UniqueName: \"kubernetes.io/projected/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-kube-api-access-2k6rr\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692860    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692865    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-host\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692886    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/96b44fa7-1fa7-4811-8513-8d367a40ed40-serviceca\") pod \"node-ca-t8ct4\" (UID: \"96b44fa7-1fa7-4811-8513-8d367a40ed40\") " pod="openshift-image-registry/node-ca-t8ct4"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692913    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692912    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2hzd\" (UniqueName: \"kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd\") pod \"network-check-target-w5mh6\" (UID: \"b0d6ff40-27c3-4a2e-9fe3-fd102227932e\") " pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692947    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-multus-socket-dir-parent\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692971    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-run-openvswitch\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.692996    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k5gp2\" (UniqueName: \"kubernetes.io/projected/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-kube-api-access-k5gp2\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693016    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-sys\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693032    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-multus-cni-dir\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693096    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-sys\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693094    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-run-openvswitch\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693218    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ndw78\" (UniqueName: \"kubernetes.io/projected/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-kube-api-access-ndw78\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.693330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693253    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693282    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/870dbc4e-8a43-4aa8-babb-64b1aa538fdd-iptables-alerter-script\") pod \"iptables-alerter-bx6zg\" (UID: \"870dbc4e-8a43-4aa8-babb-64b1aa538fdd\") " pod="openshift-network-operator/iptables-alerter-bx6zg"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693307    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a1da5c79-fc3a-4c1e-b30b-44de60231e7e-konnectivity-ca\") pod \"konnectivity-agent-wvm6p\" (UID: \"a1da5c79-fc3a-4c1e-b30b-44de60231e7e\") " pod="kube-system/konnectivity-agent-wvm6p"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693327    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/96b44fa7-1fa7-4811-8513-8d367a40ed40-serviceca\") pod \"node-ca-t8ct4\" (UID: \"96b44fa7-1fa7-4811-8513-8d367a40ed40\") " pod="openshift-image-registry/node-ca-t8ct4"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693337    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-os-release\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693362    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-run-netns\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693391    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-etc-kubernetes\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693395    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-cni-binary-copy\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693432    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693453    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-log-socket\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693489    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-cni-netd\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693493    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-log-socket\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693522    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-ovnkube-script-lib\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693547    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1-tmp-dir\") pod \"node-resolver-w7gt7\" (UID: \"d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1\") " pod="openshift-dns/node-resolver-w7gt7"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693559    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-cni-netd\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693628    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-system-cni-dir\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693678    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.694105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693706    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2zf7f\" (UniqueName: \"kubernetes.io/projected/9a900d97-0943-4380-a1e5-85bd64a30ba3-kube-api-access-2zf7f\") pod \"network-metrics-daemon-7sv8v\" (UID: \"9a900d97-0943-4380-a1e5-85bd64a30ba3\") " pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693731    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-sysctl-conf\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693743    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-system-cni-dir\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693753    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-run-ovn\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693788    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-run-ovn\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693800    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-etc-selinux\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693829    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-var-lib-kubelet\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693854    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c565637d-58a5-405b-8e81-bf2131b39c29-etc-tuned\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693877    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1-tmp-dir\") pod \"node-resolver-w7gt7\" (UID: \"d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1\") " pod="openshift-dns/node-resolver-w7gt7"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693882    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-run-multus-certs\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693911    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-kubelet\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693936    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-run-systemd\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693960    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-slash\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693970    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a1da5c79-fc3a-4c1e-b30b-44de60231e7e-konnectivity-ca\") pod \"konnectivity-agent-wvm6p\" (UID: \"a1da5c79-fc3a-4c1e-b30b-44de60231e7e\") " pod="kube-system/konnectivity-agent-wvm6p"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693985    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-var-lib-openvswitch\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693992    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-etc-selinux\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694010    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.694832 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694038    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-registration-dir\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694051    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-kubelet\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694053    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-slash\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694066    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a1da5c79-fc3a-4c1e-b30b-44de60231e7e-agent-certs\") pod \"konnectivity-agent-wvm6p\" (UID: \"a1da5c79-fc3a-4c1e-b30b-44de60231e7e\") " pod="kube-system/konnectivity-agent-wvm6p"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694084    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-run-systemd\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694090    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694092    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-sysconfig\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.693980    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/870dbc4e-8a43-4aa8-babb-64b1aa538fdd-iptables-alerter-script\") pod \"iptables-alerter-bx6zg\" (UID: \"870dbc4e-8a43-4aa8-babb-64b1aa538fdd\") " pod="openshift-network-operator/iptables-alerter-bx6zg"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694116    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-var-lib-openvswitch\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694115    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-kubernetes\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694136    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-sysctl-conf\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694154    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-sysconfig\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694142    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6clm8\" (UniqueName: \"kubernetes.io/projected/c565637d-58a5-405b-8e81-bf2131b39c29-kube-api-access-6clm8\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694163    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-kubernetes\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694193    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-var-lib-kubelet\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694190    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-ovnkube-config\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694203    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-ovnkube-script-lib\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694272    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-env-overrides\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.695541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694289    2553 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694297    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-modprobe-d\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694325    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-run-k8s-cni-cncf-io\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694334    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-registration-dir\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694360    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-var-lib-cni-multus\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694433    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-modprobe-d\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694465    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-multus-daemon-config\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694490    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-systemd-units\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694514    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-systemd-units\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694536    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-etc-openvswitch\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694559    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-sysctl-d\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694584    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-run\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694609    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c565637d-58a5-405b-8e81-bf2131b39c29-tmp\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694632    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-var-lib-kubelet\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694670    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694680    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-ovnkube-config\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694671    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-lib-modules\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.696274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694711    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-run-netns\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694720    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-etc-sysctl-d\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694727    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-run-ovn-kubernetes\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694737    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-lib-modules\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694745    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-ovn-node-metrics-cert\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694737    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-etc-openvswitch\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694761    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-run-netns\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694768    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-host-run-ovn-kubernetes\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694742    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-env-overrides\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694787    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c565637d-58a5-405b-8e81-bf2131b39c29-run\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694816    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gcd56\" (UniqueName: \"kubernetes.io/projected/d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1-kube-api-access-gcd56\") pod \"node-resolver-w7gt7\" (UID: \"d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1\") " pod="openshift-dns/node-resolver-w7gt7"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694844    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-os-release\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694892    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zqqxf\" (UniqueName: \"kubernetes.io/projected/870dbc4e-8a43-4aa8-babb-64b1aa538fdd-kube-api-access-zqqxf\") pod \"iptables-alerter-bx6zg\" (UID: \"870dbc4e-8a43-4aa8-babb-64b1aa538fdd\") " pod="openshift-network-operator/iptables-alerter-bx6zg"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694908    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-os-release\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694920    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bzjl5\" (UniqueName: \"kubernetes.io/projected/96b44fa7-1fa7-4811-8513-8d367a40ed40-kube-api-access-bzjl5\") pod \"node-ca-t8ct4\" (UID: \"96b44fa7-1fa7-4811-8513-8d367a40ed40\") " pod="openshift-image-registry/node-ca-t8ct4"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694958    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-node-log\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.694984    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-sys-fs\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.697069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.695011    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7664q\" (UniqueName: \"kubernetes.io/projected/ec97f75e-56a4-47d5-831f-e74157a2e7e2-kube-api-access-7664q\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.697575 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.695039    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.697575 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.695048    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-node-log\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.697575 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.695066    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96b44fa7-1fa7-4811-8513-8d367a40ed40-host\") pod \"node-ca-t8ct4\" (UID: \"96b44fa7-1fa7-4811-8513-8d367a40ed40\") " pod="openshift-image-registry/node-ca-t8ct4"
May 29 02:27:41.697575 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.695058    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec97f75e-56a4-47d5-831f-e74157a2e7e2-sys-fs\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.697575 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.695100    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-cnibin\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.697575 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.695126    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-multus-conf-dir\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.697575 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.695207    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96b44fa7-1fa7-4811-8513-8d367a40ed40-host\") pod \"node-ca-t8ct4\" (UID: \"96b44fa7-1fa7-4811-8513-8d367a40ed40\") " pod="openshift-image-registry/node-ca-t8ct4"
May 29 02:27:41.697575 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.695677    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.698007 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.697987    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c565637d-58a5-405b-8e81-bf2131b39c29-etc-tuned\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.698080 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.698053    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-ovn-node-metrics-cert\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.698164 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.698148    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a1da5c79-fc3a-4c1e-b30b-44de60231e7e-agent-certs\") pod \"konnectivity-agent-wvm6p\" (UID: \"a1da5c79-fc3a-4c1e-b30b-44de60231e7e\") " pod="kube-system/konnectivity-agent-wvm6p"
May 29 02:27:41.699062 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.699043    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c565637d-58a5-405b-8e81-bf2131b39c29-tmp\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.705772 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.705747    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndw78\" (UniqueName: \"kubernetes.io/projected/b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6-kube-api-access-ndw78\") pod \"ovnkube-node-2mq8b\" (UID: \"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.706773 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.706754    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5gp2\" (UniqueName: \"kubernetes.io/projected/5ef02e9b-bceb-480a-93cc-dfaaa87e8f72-kube-api-access-k5gp2\") pod \"multus-additional-cni-plugins-h6mc4\" (UID: \"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72\") " pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.710888 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.710862    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqqxf\" (UniqueName: \"kubernetes.io/projected/870dbc4e-8a43-4aa8-babb-64b1aa538fdd-kube-api-access-zqqxf\") pod \"iptables-alerter-bx6zg\" (UID: \"870dbc4e-8a43-4aa8-babb-64b1aa538fdd\") " pod="openshift-network-operator/iptables-alerter-bx6zg"
May 29 02:27:41.711349 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.711328    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zf7f\" (UniqueName: \"kubernetes.io/projected/9a900d97-0943-4380-a1e5-85bd64a30ba3-kube-api-access-2zf7f\") pod \"network-metrics-daemon-7sv8v\" (UID: \"9a900d97-0943-4380-a1e5-85bd64a30ba3\") " pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:41.711872 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.711853    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7664q\" (UniqueName: \"kubernetes.io/projected/ec97f75e-56a4-47d5-831f-e74157a2e7e2-kube-api-access-7664q\") pod \"aws-ebs-csi-driver-node-z67dx\" (UID: \"ec97f75e-56a4-47d5-831f-e74157a2e7e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.711961 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.711942    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcd56\" (UniqueName: \"kubernetes.io/projected/d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1-kube-api-access-gcd56\") pod \"node-resolver-w7gt7\" (UID: \"d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1\") " pod="openshift-dns/node-resolver-w7gt7"
May 29 02:27:41.712286 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.712268    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzjl5\" (UniqueName: \"kubernetes.io/projected/96b44fa7-1fa7-4811-8513-8d367a40ed40-kube-api-access-bzjl5\") pod \"node-ca-t8ct4\" (UID: \"96b44fa7-1fa7-4811-8513-8d367a40ed40\") " pod="openshift-image-registry/node-ca-t8ct4"
May 29 02:27:41.716997 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.716970    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6clm8\" (UniqueName: \"kubernetes.io/projected/c565637d-58a5-405b-8e81-bf2131b39c29-kube-api-access-6clm8\") pod \"tuned-qqvl8\" (UID: \"c565637d-58a5-405b-8e81-bf2131b39c29\") " pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.796073 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796040    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-var-lib-cni-multus\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796073 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796073    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-multus-daemon-config\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796246 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796090    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-var-lib-kubelet\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796246 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796113    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-cnibin\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796246 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796139    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-multus-conf-dir\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796246 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796164    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-var-lib-cni-bin\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796246 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796166    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-var-lib-cni-multus\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796246 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796202    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-system-cni-dir\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796246 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796218    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-hostroot\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796246 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796236    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-cnibin\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796246 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796247    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-cni-binary-copy\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796564 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796263    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2k6rr\" (UniqueName: \"kubernetes.io/projected/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-kube-api-access-2k6rr\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796564 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796286    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p2hzd\" (UniqueName: \"kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd\") pod \"network-check-target-w5mh6\" (UID: \"b0d6ff40-27c3-4a2e-9fe3-fd102227932e\") " pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:41.796564 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796292    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-system-cni-dir\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796564 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796308    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-multus-socket-dir-parent\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796564 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796327    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-multus-conf-dir\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796564 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796330    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-multus-cni-dir\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796564 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796375    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-os-release\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796564 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796383    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-multus-cni-dir\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796564 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796395    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-run-netns\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796564 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796419    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-hostroot\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796564 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796420    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-var-lib-cni-bin\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.796564 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796454    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-os-release\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.797077 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796688    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-multus-daemon-config\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.797077 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796737    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-etc-kubernetes\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.797077 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796265    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-var-lib-kubelet\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.797077 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796775    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-run-multus-certs\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.797077 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796807    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-run-netns\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.797077 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796812    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-run-k8s-cni-cncf-io\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.797077 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796828    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-cni-binary-copy\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.797077 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796841    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-etc-kubernetes\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.797077 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796874    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-run-k8s-cni-cncf-io\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.797077 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796884    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-host-run-multus-certs\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.797077 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.796886    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-multus-socket-dir-parent\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.804573 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:41.804512    2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 02:27:41.804573 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:41.804540    2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 02:27:41.804573 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:41.804552    2553 projected.go:194] Error preparing data for projected volume kube-api-access-p2hzd for pod openshift-network-diagnostics/network-check-target-w5mh6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:27:41.804781 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:41.804614    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd podName:b0d6ff40-27c3-4a2e-9fe3-fd102227932e nodeName:}" failed. No retries permitted until 2026-05-29 02:27:42.304600548 +0000 UTC m=+3.168213071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-p2hzd" (UniqueName: "kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd") pod "network-check-target-w5mh6" (UID: "b0d6ff40-27c3-4a2e-9fe3-fd102227932e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:27:41.806932 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.806911    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k6rr\" (UniqueName: \"kubernetes.io/projected/f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076-kube-api-access-2k6rr\") pod \"multus-t8gs7\" (UID: \"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076\") " pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.877151 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.877116    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:27:41.883112 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.883092    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx"
May 29 02:27:41.890583 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.890567    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h6mc4"
May 29 02:27:41.899245 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.899225    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bx6zg"
May 29 02:27:41.904811 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.904794    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wvm6p"
May 29 02:27:41.911300 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.911283    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qqvl8"
May 29 02:27:41.918784 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.918769    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w7gt7"
May 29 02:27:41.924298 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.924279    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t8ct4"
May 29 02:27:41.928960 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.928942    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t8gs7"
May 29 02:27:41.930694 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:41.930677    2553 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 02:27:42.095748 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:42.095715    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b44fa7_1fa7_4811_8513_8d367a40ed40.slice/crio-ca161342ac3bad1ab32e2723448d323911a8237ede008a719b2b35796a7503c8 WatchSource:0}: Error finding container ca161342ac3bad1ab32e2723448d323911a8237ede008a719b2b35796a7503c8: Status 404 returned error can't find the container with id ca161342ac3bad1ab32e2723448d323911a8237ede008a719b2b35796a7503c8
May 29 02:27:42.097780 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:42.097751    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec97f75e_56a4_47d5_831f_e74157a2e7e2.slice/crio-79675b2cf0a4f10360366a8a6368ce942dc1f1996f7f83309ac8da395adb65f6 WatchSource:0}: Error finding container 79675b2cf0a4f10360366a8a6368ce942dc1f1996f7f83309ac8da395adb65f6: Status 404 returned error can't find the container with id 79675b2cf0a4f10360366a8a6368ce942dc1f1996f7f83309ac8da395adb65f6
May 29 02:27:42.098785 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:42.098553    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef02e9b_bceb_480a_93cc_dfaaa87e8f72.slice/crio-e960347802321bb1e7af9db77a258f4c031381c54a764a29dc609a42a6505302 WatchSource:0}: Error finding container e960347802321bb1e7af9db77a258f4c031381c54a764a29dc609a42a6505302: Status 404 returned error can't find the container with id e960347802321bb1e7af9db77a258f4c031381c54a764a29dc609a42a6505302
May 29 02:27:42.101109 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:42.101081    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf05a0fdc_25f4_4b4a_a1a1_362bc7aaf076.slice/crio-e54f2d84a541ba9ca486c31ab476e2f9e7d313f70b1e6d4bf3b87a2cc4acaa99 WatchSource:0}: Error finding container e54f2d84a541ba9ca486c31ab476e2f9e7d313f70b1e6d4bf3b87a2cc4acaa99: Status 404 returned error can't find the container with id e54f2d84a541ba9ca486c31ab476e2f9e7d313f70b1e6d4bf3b87a2cc4acaa99
May 29 02:27:42.101918 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:42.101890    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd90c6dde_1b8b_4aba_9d9e_5c11f6e47dd1.slice/crio-c526196c01d221edcea8c9e28cfe2c6e9cad847d4148f0ccc56a02aa00a2f2ca WatchSource:0}: Error finding container c526196c01d221edcea8c9e28cfe2c6e9cad847d4148f0ccc56a02aa00a2f2ca: Status 404 returned error can't find the container with id c526196c01d221edcea8c9e28cfe2c6e9cad847d4148f0ccc56a02aa00a2f2ca
May 29 02:27:42.102392 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:42.102368    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc565637d_58a5_405b_8e81_bf2131b39c29.slice/crio-ece24b8b426a5c6fe04a0c80c04e94d54a8bb2d70bf760d050bfc6cf7dc91864 WatchSource:0}: Error finding container ece24b8b426a5c6fe04a0c80c04e94d54a8bb2d70bf760d050bfc6cf7dc91864: Status 404 returned error can't find the container with id ece24b8b426a5c6fe04a0c80c04e94d54a8bb2d70bf760d050bfc6cf7dc91864
May 29 02:27:42.104098 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:42.104077    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1da5c79_fc3a_4c1e_b30b_44de60231e7e.slice/crio-7db9f652671cacca72631d060d85ba594456a0305ed6aaaf7cc70799cea8b225 WatchSource:0}: Error finding container 7db9f652671cacca72631d060d85ba594456a0305ed6aaaf7cc70799cea8b225: Status 404 returned error can't find the container with id 7db9f652671cacca72631d060d85ba594456a0305ed6aaaf7cc70799cea8b225
May 29 02:27:42.104308 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:42.104247    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb44b7ac4_22d0_49f8_bde3_f8fbaa5264c6.slice/crio-ab84f361e278b433789a7b702f30822289ceb5e91a5a359bfe14048549e93ff0 WatchSource:0}: Error finding container ab84f361e278b433789a7b702f30822289ceb5e91a5a359bfe14048549e93ff0: Status 404 returned error can't find the container with id ab84f361e278b433789a7b702f30822289ceb5e91a5a359bfe14048549e93ff0
May 29 02:27:42.105464 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:27:42.105327    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod870dbc4e_8a43_4aa8_babb_64b1aa538fdd.slice/crio-9103a1b60a7fd07de784da2ea8da81280a826f59290400d3d88551f4baf983bc WatchSource:0}: Error finding container 9103a1b60a7fd07de784da2ea8da81280a826f59290400d3d88551f4baf983bc: Status 404 returned error can't find the container with id 9103a1b60a7fd07de784da2ea8da81280a826f59290400d3d88551f4baf983bc
May 29 02:27:42.199062 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.199031    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs\") pod \"network-metrics-daemon-7sv8v\" (UID: \"9a900d97-0943-4380-a1e5-85bd64a30ba3\") " pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:42.199208 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:42.199191    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:27:42.199277 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:42.199262    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs podName:9a900d97-0943-4380-a1e5-85bd64a30ba3 nodeName:}" failed. No retries permitted until 2026-05-29 02:27:43.199240625 +0000 UTC m=+4.062853150 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs") pod "network-metrics-daemon-7sv8v" (UID: "9a900d97-0943-4380-a1e5-85bd64a30ba3") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:27:42.400258 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.400235    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p2hzd\" (UniqueName: \"kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd\") pod \"network-check-target-w5mh6\" (UID: \"b0d6ff40-27c3-4a2e-9fe3-fd102227932e\") " pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:42.400375 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:42.400355    2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 02:27:42.400375 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:42.400369    2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 02:27:42.400452 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:42.400377    2553 projected.go:194] Error preparing data for projected volume kube-api-access-p2hzd for pod openshift-network-diagnostics/network-check-target-w5mh6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:27:42.400452 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:42.400424    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd podName:b0d6ff40-27c3-4a2e-9fe3-fd102227932e nodeName:}" failed. No retries permitted until 2026-05-29 02:27:43.400410229 +0000 UTC m=+4.264022739 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-p2hzd" (UniqueName: "kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd") pod "network-check-target-w5mh6" (UID: "b0d6ff40-27c3-4a2e-9fe3-fd102227932e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:27:42.644633 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.644545    2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-28 02:22:40 +0000 UTC" deadline="2027-12-04 20:01:22.498830856 +0000 UTC"
May 29 02:27:42.644633 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.644588    2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13313h33m39.854246183s"
May 29 02:27:42.716827 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.716319    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:42.716827 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:42.716447    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:27:42.733927 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.733193    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-245.ec2.internal" event={"ID":"f2cb1de6875396d67d2b36a572294b25","Type":"ContainerStarted","Data":"ce118f87060db28f85297051f3783647f692c42e7b3db0aef6fd3c1b5afe4bee"}
May 29 02:27:42.751399 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.750818    2553 generic.go:358] "Generic (PLEG): container finished" podID="5bf5e5a7948545e5d9a23a951bc43bf6" containerID="64f4d86ba102c79b399794e86cf01b6df38cba24b818a16a47f39bb8e4a30470" exitCode=0
May 29 02:27:42.751399 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.750959    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal" event={"ID":"5bf5e5a7948545e5d9a23a951bc43bf6","Type":"ContainerDied","Data":"64f4d86ba102c79b399794e86cf01b6df38cba24b818a16a47f39bb8e4a30470"}
May 29 02:27:42.753532 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.753506    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wvm6p" event={"ID":"a1da5c79-fc3a-4c1e-b30b-44de60231e7e","Type":"ContainerStarted","Data":"7db9f652671cacca72631d060d85ba594456a0305ed6aaaf7cc70799cea8b225"}
May 29 02:27:42.766915 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.766755    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t8gs7" event={"ID":"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076","Type":"ContainerStarted","Data":"e54f2d84a541ba9ca486c31ab476e2f9e7d313f70b1e6d4bf3b87a2cc4acaa99"}
May 29 02:27:42.774852 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.774826    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6mc4" event={"ID":"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72","Type":"ContainerStarted","Data":"e960347802321bb1e7af9db77a258f4c031381c54a764a29dc609a42a6505302"}
May 29 02:27:42.776303 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.776238    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-245.ec2.internal" podStartSLOduration=1.776223162 podStartE2EDuration="1.776223162s" podCreationTimestamp="2026-05-29 02:27:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 02:27:42.750680532 +0000 UTC m=+3.614293064" watchObservedRunningTime="2026-05-29 02:27:42.776223162 +0000 UTC m=+3.639835695"
May 29 02:27:42.780256 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.780236    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx" event={"ID":"ec97f75e-56a4-47d5-831f-e74157a2e7e2","Type":"ContainerStarted","Data":"79675b2cf0a4f10360366a8a6368ce942dc1f1996f7f83309ac8da395adb65f6"}
May 29 02:27:42.791041 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.790968    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bx6zg" event={"ID":"870dbc4e-8a43-4aa8-babb-64b1aa538fdd","Type":"ContainerStarted","Data":"9103a1b60a7fd07de784da2ea8da81280a826f59290400d3d88551f4baf983bc"}
May 29 02:27:42.797791 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.797757    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b" event={"ID":"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6","Type":"ContainerStarted","Data":"ab84f361e278b433789a7b702f30822289ceb5e91a5a359bfe14048549e93ff0"}
May 29 02:27:42.800272 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.800248    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w7gt7" event={"ID":"d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1","Type":"ContainerStarted","Data":"c526196c01d221edcea8c9e28cfe2c6e9cad847d4148f0ccc56a02aa00a2f2ca"}
May 29 02:27:42.803441 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.803373    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qqvl8" event={"ID":"c565637d-58a5-405b-8e81-bf2131b39c29","Type":"ContainerStarted","Data":"ece24b8b426a5c6fe04a0c80c04e94d54a8bb2d70bf760d050bfc6cf7dc91864"}
May 29 02:27:42.806210 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:42.806134    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t8ct4" event={"ID":"96b44fa7-1fa7-4811-8513-8d367a40ed40","Type":"ContainerStarted","Data":"ca161342ac3bad1ab32e2723448d323911a8237ede008a719b2b35796a7503c8"}
May 29 02:27:43.206377 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:43.206291    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs\") pod \"network-metrics-daemon-7sv8v\" (UID: \"9a900d97-0943-4380-a1e5-85bd64a30ba3\") " pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:43.206543 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:43.206471    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:27:43.206543 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:43.206532    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs podName:9a900d97-0943-4380-a1e5-85bd64a30ba3 nodeName:}" failed. No retries permitted until 2026-05-29 02:27:45.206514482 +0000 UTC m=+6.070126997 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs") pod "network-metrics-daemon-7sv8v" (UID: "9a900d97-0943-4380-a1e5-85bd64a30ba3") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:27:43.408188 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:43.408098    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p2hzd\" (UniqueName: \"kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd\") pod \"network-check-target-w5mh6\" (UID: \"b0d6ff40-27c3-4a2e-9fe3-fd102227932e\") " pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:43.408355 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:43.408266    2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 02:27:43.408355 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:43.408287    2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 02:27:43.408355 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:43.408301    2553 projected.go:194] Error preparing data for projected volume kube-api-access-p2hzd for pod openshift-network-diagnostics/network-check-target-w5mh6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:27:43.408524 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:43.408367    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd podName:b0d6ff40-27c3-4a2e-9fe3-fd102227932e nodeName:}" failed. No retries permitted until 2026-05-29 02:27:45.408345221 +0000 UTC m=+6.271957770 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-p2hzd" (UniqueName: "kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd") pod "network-check-target-w5mh6" (UID: "b0d6ff40-27c3-4a2e-9fe3-fd102227932e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:27:43.718222 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:43.718189    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:43.718677 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:43.718314    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:27:43.829209 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:43.829057    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal" event={"ID":"5bf5e5a7948545e5d9a23a951bc43bf6","Type":"ContainerStarted","Data":"0714cefc5d8fdcdb221630eb57be4bb07c7221ad6542fb61a5e6969051881a5c"}
May 29 02:27:44.716213 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:44.715713    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:44.716213 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:44.715863    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:27:45.224251 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:45.223632    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs\") pod \"network-metrics-daemon-7sv8v\" (UID: \"9a900d97-0943-4380-a1e5-85bd64a30ba3\") " pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:45.224251 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:45.223799    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:27:45.224251 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:45.223884    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs podName:9a900d97-0943-4380-a1e5-85bd64a30ba3 nodeName:}" failed. No retries permitted until 2026-05-29 02:27:49.223862593 +0000 UTC m=+10.087475123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs") pod "network-metrics-daemon-7sv8v" (UID: "9a900d97-0943-4380-a1e5-85bd64a30ba3") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:27:45.426447 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:45.425793    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p2hzd\" (UniqueName: \"kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd\") pod \"network-check-target-w5mh6\" (UID: \"b0d6ff40-27c3-4a2e-9fe3-fd102227932e\") " pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:45.426447 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:45.425999    2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 02:27:45.426447 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:45.426018    2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 02:27:45.426447 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:45.426030    2553 projected.go:194] Error preparing data for projected volume kube-api-access-p2hzd for pod openshift-network-diagnostics/network-check-target-w5mh6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:27:45.426447 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:45.426094    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd podName:b0d6ff40-27c3-4a2e-9fe3-fd102227932e nodeName:}" failed. No retries permitted until 2026-05-29 02:27:49.426075653 +0000 UTC m=+10.289688177 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-p2hzd" (UniqueName: "kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd") pod "network-check-target-w5mh6" (UID: "b0d6ff40-27c3-4a2e-9fe3-fd102227932e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:27:45.719857 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:45.719827    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:45.720044 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:45.719937    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:27:46.715927 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:46.715890    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:46.716385 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:46.716043    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:27:47.715598 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:47.715563    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:47.715925 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:47.715695    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:27:48.715751 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:48.715705    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:48.716188 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:48.715902    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:27:49.258380 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:49.258340    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs\") pod \"network-metrics-daemon-7sv8v\" (UID: \"9a900d97-0943-4380-a1e5-85bd64a30ba3\") " pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:49.258547 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:49.258532    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:27:49.258619 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:49.258611    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs podName:9a900d97-0943-4380-a1e5-85bd64a30ba3 nodeName:}" failed. No retries permitted until 2026-05-29 02:27:57.258590457 +0000 UTC m=+18.122202981 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs") pod "network-metrics-daemon-7sv8v" (UID: "9a900d97-0943-4380-a1e5-85bd64a30ba3") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:27:49.460121 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:49.460083    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p2hzd\" (UniqueName: \"kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd\") pod \"network-check-target-w5mh6\" (UID: \"b0d6ff40-27c3-4a2e-9fe3-fd102227932e\") " pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:49.460305 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:49.460249    2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 02:27:49.460305 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:49.460277    2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 02:27:49.460305 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:49.460288    2553 projected.go:194] Error preparing data for projected volume kube-api-access-p2hzd for pod openshift-network-diagnostics/network-check-target-w5mh6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:27:49.460468 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:49.460344    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd podName:b0d6ff40-27c3-4a2e-9fe3-fd102227932e nodeName:}" failed. No retries permitted until 2026-05-29 02:27:57.460329718 +0000 UTC m=+18.323942239 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-p2hzd" (UniqueName: "kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd") pod "network-check-target-w5mh6" (UID: "b0d6ff40-27c3-4a2e-9fe3-fd102227932e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:27:49.717294 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:49.716811    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:49.717294 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:49.716927    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:27:50.716222 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:50.716188    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:50.716399 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:50.716315    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:27:51.716213 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:51.716165    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:51.716699 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:51.716287    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:27:52.716167 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:52.716138    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:52.716360 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:52.716260    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:27:53.716480 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:53.716447    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:53.717014 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:53.716569    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:27:54.716584 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:54.716374    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:54.717016 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:54.716701    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:27:55.716482 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:55.716442    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:55.716664 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:55.716590    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:27:56.716534 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:56.716498    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:56.716761 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:56.716634    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:27:57.320717 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:57.320677    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs\") pod \"network-metrics-daemon-7sv8v\" (UID: \"9a900d97-0943-4380-a1e5-85bd64a30ba3\") " pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:57.320879 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:57.320837    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:27:57.320940 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:57.320905    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs podName:9a900d97-0943-4380-a1e5-85bd64a30ba3 nodeName:}" failed. No retries permitted until 2026-05-29 02:28:13.320888392 +0000 UTC m=+34.184500902 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs") pod "network-metrics-daemon-7sv8v" (UID: "9a900d97-0943-4380-a1e5-85bd64a30ba3") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:27:57.521515 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:57.521479    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p2hzd\" (UniqueName: \"kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd\") pod \"network-check-target-w5mh6\" (UID: \"b0d6ff40-27c3-4a2e-9fe3-fd102227932e\") " pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:57.521681 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:57.521619    2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 02:27:57.521681 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:57.521635    2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 02:27:57.521681 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:57.521644    2553 projected.go:194] Error preparing data for projected volume kube-api-access-p2hzd for pod openshift-network-diagnostics/network-check-target-w5mh6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:27:57.521793 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:57.521714    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd podName:b0d6ff40-27c3-4a2e-9fe3-fd102227932e nodeName:}" failed. No retries permitted until 2026-05-29 02:28:13.52169786 +0000 UTC m=+34.385310386 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-p2hzd" (UniqueName: "kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd") pod "network-check-target-w5mh6" (UID: "b0d6ff40-27c3-4a2e-9fe3-fd102227932e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:27:57.715859 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:57.715786    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:57.715999 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:57.715915    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:27:58.715876 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:58.715846    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:27:58.716291 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:58.715987    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:27:59.719519 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:27:59.718008    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:27:59.719519 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:27:59.718145    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:28:00.716980 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:00.716466    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:28:00.716980 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:00.716616    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:28:00.868159 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:00.867692    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qqvl8" event={"ID":"c565637d-58a5-405b-8e81-bf2131b39c29","Type":"ContainerStarted","Data":"00a0ad1ac547d812e21e204e4dd17813a8ac8063abbe161f49505dac9fdb69d5"}
May 29 02:28:00.869742 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:00.869455    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t8ct4" event={"ID":"96b44fa7-1fa7-4811-8513-8d367a40ed40","Type":"ContainerStarted","Data":"0647ace1563397009b60881f4e2a7d08d0fc7c097edbb14d5c8b5088cf6b02ff"}
May 29 02:28:00.872682 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:00.872579    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6mc4" event={"ID":"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72","Type":"ContainerStarted","Data":"0303332a4e43b6b3453b4da15fe68e2df3cb2e68e31d2f3af95bf66fbcbb1ccf"}
May 29 02:28:00.874390 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:00.874369    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b" event={"ID":"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6","Type":"ContainerStarted","Data":"6e338b07be3d4b39eff5734be9eee1b1f41baf3cf14993192b423716e3f3cdf6"}
May 29 02:28:00.909729 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:00.907426    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qqvl8" podStartSLOduration=3.397962786 podStartE2EDuration="21.907410774s" podCreationTimestamp="2026-05-29 02:27:39 +0000 UTC" firstStartedPulling="2026-05-29 02:27:42.104839745 +0000 UTC m=+2.968452254" lastFinishedPulling="2026-05-29 02:28:00.614287731 +0000 UTC m=+21.477900242" observedRunningTime="2026-05-29 02:28:00.905357975 +0000 UTC m=+21.768970508" watchObservedRunningTime="2026-05-29 02:28:00.907410774 +0000 UTC m=+21.771023305"
May 29 02:28:00.909729 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:00.907893    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-245.ec2.internal" podStartSLOduration=19.907881625999998 podStartE2EDuration="19.907881626s" podCreationTimestamp="2026-05-29 02:27:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 02:27:43.847672739 +0000 UTC m=+4.711285265" watchObservedRunningTime="2026-05-29 02:28:00.907881626 +0000 UTC m=+21.771494158"
May 29 02:28:01.046391 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.046339    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-t8ct4" podStartSLOduration=7.650486017 podStartE2EDuration="22.046314243s" podCreationTimestamp="2026-05-29 02:27:39 +0000 UTC" firstStartedPulling="2026-05-29 02:27:42.097613044 +0000 UTC m=+2.961225562" lastFinishedPulling="2026-05-29 02:27:56.493441277 +0000 UTC m=+17.357053788" observedRunningTime="2026-05-29 02:28:00.987857591 +0000 UTC m=+21.851470124" watchObservedRunningTime="2026-05-29 02:28:01.046314243 +0000 UTC m=+21.909926815"
May 29 02:28:01.046506 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.046441    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-t8gs7" podStartSLOduration=3.4976879 podStartE2EDuration="22.046436695s" podCreationTimestamp="2026-05-29 02:27:39 +0000 UTC" firstStartedPulling="2026-05-29 02:27:42.103558517 +0000 UTC m=+2.967171041" lastFinishedPulling="2026-05-29 02:28:00.652307327 +0000 UTC m=+21.515919836" observedRunningTime="2026-05-29 02:28:01.045786967 +0000 UTC m=+21.909399498" watchObservedRunningTime="2026-05-29 02:28:01.046436695 +0000 UTC m=+21.910049224"
May 29 02:28:01.716540 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.716300    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:28:01.716751 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:01.716645    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:28:01.826148 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.826122    2553 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 29 02:28:01.878052 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.877674    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wvm6p" event={"ID":"a1da5c79-fc3a-4c1e-b30b-44de60231e7e","Type":"ContainerStarted","Data":"baf47798bc7e6796b37faff00ff0eec5a272cba66115abe16a083e20e60a3b9b"}
May 29 02:28:01.879039 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.879004    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t8gs7" event={"ID":"f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076","Type":"ContainerStarted","Data":"f2b28a35b316cc9e4225a51ad26335d81412c4ab765eff0ebc25fa5b85b7ff5c"}
May 29 02:28:01.880380 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.880354    2553 generic.go:358] "Generic (PLEG): container finished" podID="5ef02e9b-bceb-480a-93cc-dfaaa87e8f72" containerID="0303332a4e43b6b3453b4da15fe68e2df3cb2e68e31d2f3af95bf66fbcbb1ccf" exitCode=0
May 29 02:28:01.880496 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.880421    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6mc4" event={"ID":"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72","Type":"ContainerDied","Data":"0303332a4e43b6b3453b4da15fe68e2df3cb2e68e31d2f3af95bf66fbcbb1ccf"}
May 29 02:28:01.882104 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.882083    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx" event={"ID":"ec97f75e-56a4-47d5-831f-e74157a2e7e2","Type":"ContainerStarted","Data":"35a8de0d2602c00a9fe775d25a143d40b5c8978058366635aa83dee4a9f0d96e"}
May 29 02:28:01.882204 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.882110    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx" event={"ID":"ec97f75e-56a4-47d5-831f-e74157a2e7e2","Type":"ContainerStarted","Data":"301b7ebefbf2d0d959b043e7cefc5972553f9d51368dc1bf1cc732f551cd0fef"}
May 29 02:28:01.885163 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.885142    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b" event={"ID":"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6","Type":"ContainerStarted","Data":"b0eb9ca1b2809a5077579b1a30dd37deee44b19361ff9fc07110f8865a5eea7b"}
May 29 02:28:01.885268 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.885170    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b" event={"ID":"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6","Type":"ContainerStarted","Data":"961b1d1ae3213d380ff6a7ae4bb6ddf59cb99fe2d06f96b1a04427a87ff5519a"}
May 29 02:28:01.885268 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.885193    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b" event={"ID":"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6","Type":"ContainerStarted","Data":"d9e7f976d4338eff8137a21092f1648ff005e3b161a0e7361c9df9ca1bce9ed9"}
May 29 02:28:01.885268 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.885209    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b" event={"ID":"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6","Type":"ContainerStarted","Data":"86a8bfaf47a885defbd6d08797ddaad8db036a1a2c1ff30b2db2bea053ade460"}
May 29 02:28:01.885268 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.885221    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b" event={"ID":"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6","Type":"ContainerStarted","Data":"aa25167d9afcc2bd28eba18ac7c7f91f84a679b744e6925cfd2b5c3cf14222aa"}
May 29 02:28:01.886676 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.886345    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w7gt7" event={"ID":"d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1","Type":"ContainerStarted","Data":"9cec6048293c277e7477cb57ac3692b4472566bab6d8c1108d3dc349c18d7c9d"}
May 29 02:28:01.904045 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:01.904000    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wvm6p" podStartSLOduration=4.400100809 podStartE2EDuration="22.903990029s" podCreationTimestamp="2026-05-29 02:27:39 +0000 UTC" firstStartedPulling="2026-05-29 02:27:42.10775466 +0000 UTC m=+2.971367183" lastFinishedPulling="2026-05-29 02:28:00.611643887 +0000 UTC m=+21.475256403" observedRunningTime="2026-05-29 02:28:01.903789642 +0000 UTC m=+22.767402175" watchObservedRunningTime="2026-05-29 02:28:01.903990029 +0000 UTC m=+22.767602560"
May 29 02:28:02.650967 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:02.650865    2553 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-29T02:28:01.826144659Z","UUID":"1b464a81-24d7-49d9-8b18-e2bb7bb22a47","Handler":null,"Name":"","Endpoint":""}
May 29 02:28:02.652640 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:02.652618    2553 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 29 02:28:02.652756 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:02.652670    2553 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 29 02:28:02.716517 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:02.716475    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:28:02.716678 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:02.716602    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:28:02.890256 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:02.890171    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx" event={"ID":"ec97f75e-56a4-47d5-831f-e74157a2e7e2","Type":"ContainerStarted","Data":"ff844191af7bf2309af01d8b2f4b8016f82620485a0289d4da190ca8976ed571"}
May 29 02:28:02.891619 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:02.891572    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bx6zg" event={"ID":"870dbc4e-8a43-4aa8-babb-64b1aa538fdd","Type":"ContainerStarted","Data":"f31ec4392e8669925ae3af85e832683d1043cf957f9418c78ebb315ec07b0f98"}
May 29 02:28:02.909308 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:02.909270    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-w7gt7" podStartSLOduration=5.401328932 podStartE2EDuration="23.909258287s" podCreationTimestamp="2026-05-29 02:27:39 +0000 UTC" firstStartedPulling="2026-05-29 02:27:42.103914059 +0000 UTC m=+2.967526583" lastFinishedPulling="2026-05-29 02:28:00.611843414 +0000 UTC m=+21.475455938" observedRunningTime="2026-05-29 02:28:01.923941596 +0000 UTC m=+22.787554129" watchObservedRunningTime="2026-05-29 02:28:02.909258287 +0000 UTC m=+23.772870867"
May 29 02:28:02.909511 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:02.909490    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z67dx" podStartSLOduration=3.432256833 podStartE2EDuration="23.909485609s" podCreationTimestamp="2026-05-29 02:27:39 +0000 UTC" firstStartedPulling="2026-05-29 02:27:42.099786785 +0000 UTC m=+2.963399310" lastFinishedPulling="2026-05-29 02:28:02.577015573 +0000 UTC m=+23.440628086" observedRunningTime="2026-05-29 02:28:02.908795359 +0000 UTC m=+23.772407892" watchObservedRunningTime="2026-05-29 02:28:02.909485609 +0000 UTC m=+23.773098141"
May 29 02:28:02.929027 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:02.928984    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bx6zg" podStartSLOduration=5.426051137 podStartE2EDuration="23.928972225s" podCreationTimestamp="2026-05-29 02:27:39 +0000 UTC" firstStartedPulling="2026-05-29 02:27:42.108920266 +0000 UTC m=+2.972532777" lastFinishedPulling="2026-05-29 02:28:00.61184134 +0000 UTC m=+21.475453865" observedRunningTime="2026-05-29 02:28:02.92869317 +0000 UTC m=+23.792305699" watchObservedRunningTime="2026-05-29 02:28:02.928972225 +0000 UTC m=+23.792584756"
May 29 02:28:03.716158 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:03.716123    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:28:03.716322 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:03.716249    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:28:04.716590 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:04.716560    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:28:04.717143 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:04.716692    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:28:04.756502 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:04.756470    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wvm6p"
May 29 02:28:04.757123 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:04.757103    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wvm6p"
May 29 02:28:04.898457 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:04.898422    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b" event={"ID":"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6","Type":"ContainerStarted","Data":"70bf1e4f001b7bc76fb93282a05a78052a0257e73e77bd152b9d6ff7380a666d"}
May 29 02:28:04.898627 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:04.898585    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wvm6p"
May 29 02:28:04.899119 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:04.899095    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wvm6p"
May 29 02:28:05.715761 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:05.715734    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:28:05.715896 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:05.715850    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:28:05.901710 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:05.901678    2553 generic.go:358] "Generic (PLEG): container finished" podID="5ef02e9b-bceb-480a-93cc-dfaaa87e8f72" containerID="81239e060bb16e68ad922166266721ac7098dbac1b2f615f04b6a8877868dfdf" exitCode=0
May 29 02:28:05.902067 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:05.901771    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6mc4" event={"ID":"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72","Type":"ContainerDied","Data":"81239e060bb16e68ad922166266721ac7098dbac1b2f615f04b6a8877868dfdf"}
May 29 02:28:06.715852 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:06.715819    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:28:06.716031 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:06.715960    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:28:07.716825 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:07.716628    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:28:07.717300 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:07.716887    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:28:07.906457 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:07.906423    2553 generic.go:358] "Generic (PLEG): container finished" podID="5ef02e9b-bceb-480a-93cc-dfaaa87e8f72" containerID="4a2956334cf0369bc92267c8cdd3a80bd7c08438ae85531ef0baed78b38a2595" exitCode=0
May 29 02:28:07.906589 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:07.906502    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6mc4" event={"ID":"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72","Type":"ContainerDied","Data":"4a2956334cf0369bc92267c8cdd3a80bd7c08438ae85531ef0baed78b38a2595"}
May 29 02:28:07.909886 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:07.909868    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b" event={"ID":"b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6","Type":"ContainerStarted","Data":"769a201fcea8760fd0dca833c3ebaab8d5622564d8c8bdcea95d5a7548429064"}
May 29 02:28:07.910196 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:07.910178    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:28:07.910250 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:07.910209    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:28:07.926218 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:07.926195    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:28:07.972384 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:07.972302    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b" podStartSLOduration=10.441077028 podStartE2EDuration="28.972287834s" podCreationTimestamp="2026-05-29 02:27:39 +0000 UTC" firstStartedPulling="2026-05-29 02:27:42.108855201 +0000 UTC m=+2.972467711" lastFinishedPulling="2026-05-29 02:28:00.640066006 +0000 UTC m=+21.503678517" observedRunningTime="2026-05-29 02:28:07.971366859 +0000 UTC m=+28.834979390" watchObservedRunningTime="2026-05-29 02:28:07.972287834 +0000 UTC m=+28.835900396"
May 29 02:28:08.716523 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:08.716484    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:28:08.716645 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:08.716606    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:28:08.916675 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:08.916452    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:28:08.936249 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:08.936222    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:28:09.075311 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:09.075272    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7sv8v"]
May 29 02:28:09.075444 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:09.075425    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:28:09.075583 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:09.075560    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:28:09.076113 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:09.076083    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-w5mh6"]
May 29 02:28:09.076231 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:09.076193    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:28:09.076310 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:09.076286    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:28:09.920397 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:09.920359    2553 generic.go:358] "Generic (PLEG): container finished" podID="5ef02e9b-bceb-480a-93cc-dfaaa87e8f72" containerID="25b9b580e9f61a752b3d8d10744db7c50e925a07fd9bcf7972af88a0e2110128" exitCode=0
May 29 02:28:09.921157 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:09.920443    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6mc4" event={"ID":"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72","Type":"ContainerDied","Data":"25b9b580e9f61a752b3d8d10744db7c50e925a07fd9bcf7972af88a0e2110128"}
May 29 02:28:10.715898 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:10.715865    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:28:10.716067 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:10.715975    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:28:10.716067 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:10.716028    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:28:10.716196 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:10.716165    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:28:12.716424 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:12.716390    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:28:12.717117 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:12.716394    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:28:12.717117 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:12.716530    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w5mh6" podUID="b0d6ff40-27c3-4a2e-9fe3-fd102227932e"
May 29 02:28:12.717117 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:12.716599    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:28:13.344447 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.344419    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs\") pod \"network-metrics-daemon-7sv8v\" (UID: \"9a900d97-0943-4380-a1e5-85bd64a30ba3\") " pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:28:13.344601 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:13.344539    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:28:13.344601 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:13.344590    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs podName:9a900d97-0943-4380-a1e5-85bd64a30ba3 nodeName:}" failed. No retries permitted until 2026-05-29 02:28:45.34457625 +0000 UTC m=+66.208188760 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs") pod "network-metrics-daemon-7sv8v" (UID: "9a900d97-0943-4380-a1e5-85bd64a30ba3") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 02:28:13.431377 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.431305    2553 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-245.ec2.internal" event="NodeReady"
May 29 02:28:13.431521 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.431464    2553 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 29 02:28:13.488222 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.488187    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fzhb8"]
May 29 02:28:13.522363 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.522326    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-d4shx"]
May 29 02:28:13.522528 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.522508    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:28:13.525155 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.525131    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 29 02:28:13.525721 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.525476    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jrg6b\""
May 29 02:28:13.525721 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.525540    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 29 02:28:13.525721 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.525473    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 29 02:28:13.535327 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.535308    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fzhb8"]
May 29 02:28:13.535419 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.535334    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-d4shx"]
May 29 02:28:13.535463 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.535437    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-d4shx"
May 29 02:28:13.538746 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.538728    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 29 02:28:13.539498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.539478    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zl2l4\""
May 29 02:28:13.540014 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.539989    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 29 02:28:13.545687 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.545643    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p2hzd\" (UniqueName: \"kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd\") pod \"network-check-target-w5mh6\" (UID: \"b0d6ff40-27c3-4a2e-9fe3-fd102227932e\") " pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:28:13.545781 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:13.545763    2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 02:28:13.545781 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:13.545780    2553 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 02:28:13.545870 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:13.545791    2553 projected.go:194] Error preparing data for projected volume kube-api-access-p2hzd for pod openshift-network-diagnostics/network-check-target-w5mh6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:28:13.545870 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:13.545853    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd podName:b0d6ff40-27c3-4a2e-9fe3-fd102227932e nodeName:}" failed. No retries permitted until 2026-05-29 02:28:45.545839416 +0000 UTC m=+66.409451926 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-p2hzd" (UniqueName: "kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd") pod "network-check-target-w5mh6" (UID: "b0d6ff40-27c3-4a2e-9fe3-fd102227932e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 02:28:13.646962 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.646917    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5qqt\" (UniqueName: \"kubernetes.io/projected/2a4ccab7-f230-42f0-86ba-97f21234b006-kube-api-access-w5qqt\") pod \"ingress-canary-fzhb8\" (UID: \"2a4ccab7-f230-42f0-86ba-97f21234b006\") " pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:28:13.647143 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.646974    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfv6x\" (UniqueName: \"kubernetes.io/projected/21d4bc93-ef47-4e84-b0ea-4c77774456ea-kube-api-access-nfv6x\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:13.647143 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.647005    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/21d4bc93-ef47-4e84-b0ea-4c77774456ea-config-volume\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:13.647143 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.647036    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21d4bc93-ef47-4e84-b0ea-4c77774456ea-tmp-dir\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:13.647295 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.647191    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert\") pod \"ingress-canary-fzhb8\" (UID: \"2a4ccab7-f230-42f0-86ba-97f21234b006\") " pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:28:13.647295 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.647231    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:13.747783 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.747702    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert\") pod \"ingress-canary-fzhb8\" (UID: \"2a4ccab7-f230-42f0-86ba-97f21234b006\") " pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:28:13.747783 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.747740    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:13.747783 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.747774    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w5qqt\" (UniqueName: \"kubernetes.io/projected/2a4ccab7-f230-42f0-86ba-97f21234b006-kube-api-access-w5qqt\") pod \"ingress-canary-fzhb8\" (UID: \"2a4ccab7-f230-42f0-86ba-97f21234b006\") " pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:28:13.748264 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:13.747868    2553 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 02:28:13.748264 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.747904    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nfv6x\" (UniqueName: \"kubernetes.io/projected/21d4bc93-ef47-4e84-b0ea-4c77774456ea-kube-api-access-nfv6x\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:13.748264 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:13.747937    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert podName:2a4ccab7-f230-42f0-86ba-97f21234b006 nodeName:}" failed. No retries permitted until 2026-05-29 02:28:14.247915209 +0000 UTC m=+35.111527738 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert") pod "ingress-canary-fzhb8" (UID: "2a4ccab7-f230-42f0-86ba-97f21234b006") : secret "canary-serving-cert" not found
May 29 02:28:13.748264 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:13.747870    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 02:28:13.748264 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.747970    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/21d4bc93-ef47-4e84-b0ea-4c77774456ea-config-volume\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:13.748264 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:13.748026    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls podName:21d4bc93-ef47-4e84-b0ea-4c77774456ea nodeName:}" failed. No retries permitted until 2026-05-29 02:28:14.247994192 +0000 UTC m=+35.111606705 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls") pod "dns-default-d4shx" (UID: "21d4bc93-ef47-4e84-b0ea-4c77774456ea") : secret "dns-default-metrics-tls" not found
May 29 02:28:13.748264 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.748052    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21d4bc93-ef47-4e84-b0ea-4c77774456ea-tmp-dir\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:13.748498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.748322    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21d4bc93-ef47-4e84-b0ea-4c77774456ea-tmp-dir\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:13.748554 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.748519    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/21d4bc93-ef47-4e84-b0ea-4c77774456ea-config-volume\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:13.759101 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.758927    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfv6x\" (UniqueName: \"kubernetes.io/projected/21d4bc93-ef47-4e84-b0ea-4c77774456ea-kube-api-access-nfv6x\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:13.759229 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:13.759007    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5qqt\" (UniqueName: \"kubernetes.io/projected/2a4ccab7-f230-42f0-86ba-97f21234b006-kube-api-access-w5qqt\") pod \"ingress-canary-fzhb8\" (UID: \"2a4ccab7-f230-42f0-86ba-97f21234b006\") " pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:28:14.250963 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.250918    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert\") pod \"ingress-canary-fzhb8\" (UID: \"2a4ccab7-f230-42f0-86ba-97f21234b006\") " pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:28:14.251188 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.250981    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:14.251188 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:14.251073    2553 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 02:28:14.251188 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:14.251117    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 02:28:14.251188 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:14.251174    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls podName:21d4bc93-ef47-4e84-b0ea-4c77774456ea nodeName:}" failed. No retries permitted until 2026-05-29 02:28:15.251153686 +0000 UTC m=+36.114766209 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls") pod "dns-default-d4shx" (UID: "21d4bc93-ef47-4e84-b0ea-4c77774456ea") : secret "dns-default-metrics-tls" not found
May 29 02:28:14.251188 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:14.251192    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert podName:2a4ccab7-f230-42f0-86ba-97f21234b006 nodeName:}" failed. No retries permitted until 2026-05-29 02:28:15.25118314 +0000 UTC m=+36.114795651 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert") pod "ingress-canary-fzhb8" (UID: "2a4ccab7-f230-42f0-86ba-97f21234b006") : secret "canary-serving-cert" not found
May 29 02:28:14.715873 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.715837    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:28:14.716068 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.715845    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:28:14.718876 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.718850    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 29 02:28:14.719137 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.719115    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jrq9j\""
May 29 02:28:14.720000 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.719975    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 29 02:28:14.720000 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.719996    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 29 02:28:14.720206 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.720191    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t2rwn\""
May 29 02:28:14.891449 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.891422    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65674dc458-wggjj"]
May 29 02:28:14.923061 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.922958    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65674dc458-wggjj"]
May 29 02:28:14.923061 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.922996    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"]
May 29 02:28:14.923262 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.923148    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65674dc458-wggjj"
May 29 02:28:14.926290 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.926266    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
May 29 02:28:14.926419 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.926266    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
May 29 02:28:14.926520 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.926501    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-xgbcc\""
May 29 02:28:14.926585 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.926278    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
May 29 02:28:14.926585 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.926538    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
May 29 02:28:14.941641 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.941618    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:14.944049 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.944032    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
May 29 02:28:14.944328 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.944142    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
May 29 02:28:14.944328 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.944238    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
May 29 02:28:14.944328 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.944291    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
May 29 02:28:14.945498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:14.945484    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"]
May 29 02:28:15.056334 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.056291    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63b6ddb5-4ab9-41c7-8687-54230ad2de82-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65674dc458-wggjj\" (UID: \"63b6ddb5-4ab9-41c7-8687-54230ad2de82\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65674dc458-wggjj"
May 29 02:28:15.056487 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.056347    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/88059012-e845-47ed-96e2-4b4262703155-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.056487 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.056436    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/88059012-e845-47ed-96e2-4b4262703155-hub\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.056487 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.056462    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rvlc\" (UniqueName: \"kubernetes.io/projected/63b6ddb5-4ab9-41c7-8687-54230ad2de82-kube-api-access-7rvlc\") pod \"managed-serviceaccount-addon-agent-65674dc458-wggjj\" (UID: \"63b6ddb5-4ab9-41c7-8687-54230ad2de82\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65674dc458-wggjj"
May 29 02:28:15.056613 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.056516    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88059012-e845-47ed-96e2-4b4262703155-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.056679 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.056613    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/88059012-e845-47ed-96e2-4b4262703155-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.056679 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.056639    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/88059012-e845-47ed-96e2-4b4262703155-ca\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.056802 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.056679    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkjt2\" (UniqueName: \"kubernetes.io/projected/88059012-e845-47ed-96e2-4b4262703155-kube-api-access-bkjt2\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.157254 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.157218    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/88059012-e845-47ed-96e2-4b4262703155-ca\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.157254 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.157264    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bkjt2\" (UniqueName: \"kubernetes.io/projected/88059012-e845-47ed-96e2-4b4262703155-kube-api-access-bkjt2\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.157607 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.157590    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63b6ddb5-4ab9-41c7-8687-54230ad2de82-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65674dc458-wggjj\" (UID: \"63b6ddb5-4ab9-41c7-8687-54230ad2de82\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65674dc458-wggjj"
May 29 02:28:15.157726 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.157626    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/88059012-e845-47ed-96e2-4b4262703155-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.157726 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.157688    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/88059012-e845-47ed-96e2-4b4262703155-hub\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.157726 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.157713    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7rvlc\" (UniqueName: \"kubernetes.io/projected/63b6ddb5-4ab9-41c7-8687-54230ad2de82-kube-api-access-7rvlc\") pod \"managed-serviceaccount-addon-agent-65674dc458-wggjj\" (UID: \"63b6ddb5-4ab9-41c7-8687-54230ad2de82\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65674dc458-wggjj"
May 29 02:28:15.157874 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.157761    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88059012-e845-47ed-96e2-4b4262703155-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.157874 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.157828    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/88059012-e845-47ed-96e2-4b4262703155-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.158433 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.158399    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/88059012-e845-47ed-96e2-4b4262703155-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.160993 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.160966    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/88059012-e845-47ed-96e2-4b4262703155-ca\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.160993 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.160978    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/88059012-e845-47ed-96e2-4b4262703155-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.161344 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.161324    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63b6ddb5-4ab9-41c7-8687-54230ad2de82-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65674dc458-wggjj\" (UID: \"63b6ddb5-4ab9-41c7-8687-54230ad2de82\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65674dc458-wggjj"
May 29 02:28:15.161462 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.161446    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/88059012-e845-47ed-96e2-4b4262703155-hub\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.161581 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.161566    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88059012-e845-47ed-96e2-4b4262703155-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.167178 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.167158    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkjt2\" (UniqueName: \"kubernetes.io/projected/88059012-e845-47ed-96e2-4b4262703155-kube-api-access-bkjt2\") pod \"cluster-proxy-proxy-agent-69d8fd48b6-2hkq9\" (UID: \"88059012-e845-47ed-96e2-4b4262703155\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.167307 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.167289    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rvlc\" (UniqueName: \"kubernetes.io/projected/63b6ddb5-4ab9-41c7-8687-54230ad2de82-kube-api-access-7rvlc\") pod \"managed-serviceaccount-addon-agent-65674dc458-wggjj\" (UID: \"63b6ddb5-4ab9-41c7-8687-54230ad2de82\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65674dc458-wggjj"
May 29 02:28:15.241820 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.241784    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65674dc458-wggjj"
May 29 02:28:15.251741 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.251718    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:28:15.258595 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.258568    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert\") pod \"ingress-canary-fzhb8\" (UID: \"2a4ccab7-f230-42f0-86ba-97f21234b006\") " pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:28:15.258735 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.258604    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:15.258796 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:15.258740    2553 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 02:28:15.258796 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:15.258749    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 02:28:15.258859 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:15.258808    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert podName:2a4ccab7-f230-42f0-86ba-97f21234b006 nodeName:}" failed. No retries permitted until 2026-05-29 02:28:17.258792727 +0000 UTC m=+38.122405237 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert") pod "ingress-canary-fzhb8" (UID: "2a4ccab7-f230-42f0-86ba-97f21234b006") : secret "canary-serving-cert" not found
May 29 02:28:15.258859 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:15.258823    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls podName:21d4bc93-ef47-4e84-b0ea-4c77774456ea nodeName:}" failed. No retries permitted until 2026-05-29 02:28:17.258816827 +0000 UTC m=+38.122429337 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls") pod "dns-default-d4shx" (UID: "21d4bc93-ef47-4e84-b0ea-4c77774456ea") : secret "dns-default-metrics-tls" not found
May 29 02:28:15.756055 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.756006    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65674dc458-wggjj"]
May 29 02:28:15.759052 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.759029    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"]
May 29 02:28:15.847638 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:28:15.847567    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63b6ddb5_4ab9_41c7_8687_54230ad2de82.slice/crio-8e7d3eeb09053fa75425afb45196c4058e0889e6ad03a2ea9e5648fbb07529ec WatchSource:0}: Error finding container 8e7d3eeb09053fa75425afb45196c4058e0889e6ad03a2ea9e5648fbb07529ec: Status 404 returned error can't find the container with id 8e7d3eeb09053fa75425afb45196c4058e0889e6ad03a2ea9e5648fbb07529ec
May 29 02:28:15.847973 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:28:15.847952    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88059012_e845_47ed_96e2_4b4262703155.slice/crio-529d67aa18766ed8ae7c95421b0d7b0f19860ba899e0cc15772e158bf82e172b WatchSource:0}: Error finding container 529d67aa18766ed8ae7c95421b0d7b0f19860ba899e0cc15772e158bf82e172b: Status 404 returned error can't find the container with id 529d67aa18766ed8ae7c95421b0d7b0f19860ba899e0cc15772e158bf82e172b
May 29 02:28:15.932069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.932040    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65674dc458-wggjj" event={"ID":"63b6ddb5-4ab9-41c7-8687-54230ad2de82","Type":"ContainerStarted","Data":"8e7d3eeb09053fa75425afb45196c4058e0889e6ad03a2ea9e5648fbb07529ec"}
May 29 02:28:15.933017 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:15.932990    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9" event={"ID":"88059012-e845-47ed-96e2-4b4262703155","Type":"ContainerStarted","Data":"529d67aa18766ed8ae7c95421b0d7b0f19860ba899e0cc15772e158bf82e172b"}
May 29 02:28:16.937774 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:16.937732    2553 generic.go:358] "Generic (PLEG): container finished" podID="5ef02e9b-bceb-480a-93cc-dfaaa87e8f72" containerID="be8a1520bb88c89f3fde49d6466788900c162ee5b9cf631825f5c3d77b96c7e6" exitCode=0
May 29 02:28:16.938354 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:16.937789    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6mc4" event={"ID":"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72","Type":"ContainerDied","Data":"be8a1520bb88c89f3fde49d6466788900c162ee5b9cf631825f5c3d77b96c7e6"}
May 29 02:28:17.276082 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:17.275852    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert\") pod \"ingress-canary-fzhb8\" (UID: \"2a4ccab7-f230-42f0-86ba-97f21234b006\") " pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:28:17.276255 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:17.276094    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:17.276255 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:17.276182    2553 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 02:28:17.276364 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:17.276256    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 02:28:17.276364 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:17.276266    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert podName:2a4ccab7-f230-42f0-86ba-97f21234b006 nodeName:}" failed. No retries permitted until 2026-05-29 02:28:21.27624434 +0000 UTC m=+42.139856850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert") pod "ingress-canary-fzhb8" (UID: "2a4ccab7-f230-42f0-86ba-97f21234b006") : secret "canary-serving-cert" not found
May 29 02:28:17.276364 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:17.276308    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls podName:21d4bc93-ef47-4e84-b0ea-4c77774456ea nodeName:}" failed. No retries permitted until 2026-05-29 02:28:21.276291915 +0000 UTC m=+42.139904440 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls") pod "dns-default-d4shx" (UID: "21d4bc93-ef47-4e84-b0ea-4c77774456ea") : secret "dns-default-metrics-tls" not found
May 29 02:28:17.946713 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:17.946680    2553 generic.go:358] "Generic (PLEG): container finished" podID="5ef02e9b-bceb-480a-93cc-dfaaa87e8f72" containerID="d4122d344d90c0bca7a34db51db1a505c2a2dba69d4b4f6feda5d7fc7bf93614" exitCode=0
May 29 02:28:17.947150 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:17.946758    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6mc4" event={"ID":"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72","Type":"ContainerDied","Data":"d4122d344d90c0bca7a34db51db1a505c2a2dba69d4b4f6feda5d7fc7bf93614"}
May 29 02:28:20.955644 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:20.955612    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6mc4" event={"ID":"5ef02e9b-bceb-480a-93cc-dfaaa87e8f72","Type":"ContainerStarted","Data":"a570ad5ba0e73d015eed22d9bbadbb44a3391f8c23b2e04e16cc673b18d76d9d"}
May 29 02:28:20.956906 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:20.956885    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9" event={"ID":"88059012-e845-47ed-96e2-4b4262703155","Type":"ContainerStarted","Data":"b1f2abec6e80075ea7cf7e09c2878a70356fba39f2d9f6035615968d7a62bb12"}
May 29 02:28:20.958084 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:20.958062    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65674dc458-wggjj" event={"ID":"63b6ddb5-4ab9-41c7-8687-54230ad2de82","Type":"ContainerStarted","Data":"83d85bd29649ed31ee8be6cbd60083cbde12fa197c4190ef2640118399d35c2e"}
May 29 02:28:20.982281 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:20.982232    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-h6mc4" podStartSLOduration=8.206536838 podStartE2EDuration="41.982218865s" podCreationTimestamp="2026-05-29 02:27:39 +0000 UTC" firstStartedPulling="2026-05-29 02:27:42.10108727 +0000 UTC m=+2.964699785" lastFinishedPulling="2026-05-29 02:28:15.876769302 +0000 UTC m=+36.740381812" observedRunningTime="2026-05-29 02:28:20.980869899 +0000 UTC m=+41.844482432" watchObservedRunningTime="2026-05-29 02:28:20.982218865 +0000 UTC m=+41.845831397"
May 29 02:28:20.996199 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:20.996163    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65674dc458-wggjj" podStartSLOduration=2.723592527 podStartE2EDuration="6.996154749s" podCreationTimestamp="2026-05-29 02:28:14 +0000 UTC" firstStartedPulling="2026-05-29 02:28:15.852588593 +0000 UTC m=+36.716201103" lastFinishedPulling="2026-05-29 02:28:20.125150813 +0000 UTC m=+40.988763325" observedRunningTime="2026-05-29 02:28:20.995784018 +0000 UTC m=+41.859396551" watchObservedRunningTime="2026-05-29 02:28:20.996154749 +0000 UTC m=+41.859767281"
May 29 02:28:21.309842 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:21.309801    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert\") pod \"ingress-canary-fzhb8\" (UID: \"2a4ccab7-f230-42f0-86ba-97f21234b006\") " pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:28:21.310022 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:21.309858    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:21.310022 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:21.309977    2553 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 02:28:21.310123 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:21.310038    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert podName:2a4ccab7-f230-42f0-86ba-97f21234b006 nodeName:}" failed. No retries permitted until 2026-05-29 02:28:29.310023056 +0000 UTC m=+50.173635579 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert") pod "ingress-canary-fzhb8" (UID: "2a4ccab7-f230-42f0-86ba-97f21234b006") : secret "canary-serving-cert" not found
May 29 02:28:21.310123 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:21.309979    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 02:28:21.310123 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:21.310120    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls podName:21d4bc93-ef47-4e84-b0ea-4c77774456ea nodeName:}" failed. No retries permitted until 2026-05-29 02:28:29.310105798 +0000 UTC m=+50.173718314 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls") pod "dns-default-d4shx" (UID: "21d4bc93-ef47-4e84-b0ea-4c77774456ea") : secret "dns-default-metrics-tls" not found
May 29 02:28:23.968159 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:23.968125    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9" event={"ID":"88059012-e845-47ed-96e2-4b4262703155","Type":"ContainerStarted","Data":"8600b81e01adbb5685c50675bffbf68271377c3c4ffb8e70e969a043de57cb25"}
May 29 02:28:23.968159 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:23.968162    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9" event={"ID":"88059012-e845-47ed-96e2-4b4262703155","Type":"ContainerStarted","Data":"37e47819bf42f73c90834c56853b5083176c91860eb5c591763fcb1abec57f1f"}
May 29 02:28:23.990908 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:23.990867    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9" podStartSLOduration=2.852156841 podStartE2EDuration="9.990854243s" podCreationTimestamp="2026-05-29 02:28:14 +0000 UTC" firstStartedPulling="2026-05-29 02:28:15.852417678 +0000 UTC m=+36.716030191" lastFinishedPulling="2026-05-29 02:28:22.991115083 +0000 UTC m=+43.854727593" observedRunningTime="2026-05-29 02:28:23.989478214 +0000 UTC m=+44.853090746" watchObservedRunningTime="2026-05-29 02:28:23.990854243 +0000 UTC m=+44.854466774"
May 29 02:28:29.367205 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:29.367168    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert\") pod \"ingress-canary-fzhb8\" (UID: \"2a4ccab7-f230-42f0-86ba-97f21234b006\") " pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:28:29.367205 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:29.367204    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:29.367706 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:29.367307    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 02:28:29.367706 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:29.367312    2553 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 02:28:29.367706 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:29.367361    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls podName:21d4bc93-ef47-4e84-b0ea-4c77774456ea nodeName:}" failed. No retries permitted until 2026-05-29 02:28:45.367347777 +0000 UTC m=+66.230960286 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls") pod "dns-default-d4shx" (UID: "21d4bc93-ef47-4e84-b0ea-4c77774456ea") : secret "dns-default-metrics-tls" not found
May 29 02:28:29.367706 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:29.367381    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert podName:2a4ccab7-f230-42f0-86ba-97f21234b006 nodeName:}" failed. No retries permitted until 2026-05-29 02:28:45.367366671 +0000 UTC m=+66.230979198 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert") pod "ingress-canary-fzhb8" (UID: "2a4ccab7-f230-42f0-86ba-97f21234b006") : secret "canary-serving-cert" not found
May 29 02:28:40.933439 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:40.933407    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2mq8b"
May 29 02:28:45.372573 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:45.372515    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs\") pod \"network-metrics-daemon-7sv8v\" (UID: \"9a900d97-0943-4380-a1e5-85bd64a30ba3\") " pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:28:45.373043 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:45.372591    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert\") pod \"ingress-canary-fzhb8\" (UID: \"2a4ccab7-f230-42f0-86ba-97f21234b006\") " pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:28:45.373043 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:45.372612    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:28:45.373043 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:45.372714    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 02:28:45.373043 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:45.372753    2553 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 02:28:45.373043 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:45.372785    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls podName:21d4bc93-ef47-4e84-b0ea-4c77774456ea nodeName:}" failed. No retries permitted until 2026-05-29 02:29:17.372768424 +0000 UTC m=+98.236380954 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls") pod "dns-default-d4shx" (UID: "21d4bc93-ef47-4e84-b0ea-4c77774456ea") : secret "dns-default-metrics-tls" not found
May 29 02:28:45.373043 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:45.372815    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert podName:2a4ccab7-f230-42f0-86ba-97f21234b006 nodeName:}" failed. No retries permitted until 2026-05-29 02:29:17.372797904 +0000 UTC m=+98.236410431 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert") pod "ingress-canary-fzhb8" (UID: "2a4ccab7-f230-42f0-86ba-97f21234b006") : secret "canary-serving-cert" not found
May 29 02:28:45.375529 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:45.375511    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 29 02:28:45.382728 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:45.382714    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 29 02:28:45.382774 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:28:45.382755    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs podName:9a900d97-0943-4380-a1e5-85bd64a30ba3 nodeName:}" failed. No retries permitted until 2026-05-29 02:29:49.382743574 +0000 UTC m=+130.246356084 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs") pod "network-metrics-daemon-7sv8v" (UID: "9a900d97-0943-4380-a1e5-85bd64a30ba3") : secret "metrics-daemon-secret" not found
May 29 02:28:45.574065 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:45.574030    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p2hzd\" (UniqueName: \"kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd\") pod \"network-check-target-w5mh6\" (UID: \"b0d6ff40-27c3-4a2e-9fe3-fd102227932e\") " pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:28:45.577394 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:45.577377    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 29 02:28:45.587341 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:45.587325    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 29 02:28:45.598401 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:45.598382    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2hzd\" (UniqueName: \"kubernetes.io/projected/b0d6ff40-27c3-4a2e-9fe3-fd102227932e-kube-api-access-p2hzd\") pod \"network-check-target-w5mh6\" (UID: \"b0d6ff40-27c3-4a2e-9fe3-fd102227932e\") " pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:28:45.631091 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:45.631041    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jrq9j\""
May 29 02:28:45.639308 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:45.639289    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:28:45.757245 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:45.757214    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-w5mh6"]
May 29 02:28:45.760373 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:28:45.760334    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0d6ff40_27c3_4a2e_9fe3_fd102227932e.slice/crio-2675a494bb6c88958e4a6167c79f05214a90fd45ec7d97e4a1e79a4d58c89ca5 WatchSource:0}: Error finding container 2675a494bb6c88958e4a6167c79f05214a90fd45ec7d97e4a1e79a4d58c89ca5: Status 404 returned error can't find the container with id 2675a494bb6c88958e4a6167c79f05214a90fd45ec7d97e4a1e79a4d58c89ca5
May 29 02:28:46.010433 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:46.010352    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-w5mh6" event={"ID":"b0d6ff40-27c3-4a2e-9fe3-fd102227932e","Type":"ContainerStarted","Data":"2675a494bb6c88958e4a6167c79f05214a90fd45ec7d97e4a1e79a4d58c89ca5"}
May 29 02:28:49.017743 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:49.017703    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-w5mh6" event={"ID":"b0d6ff40-27c3-4a2e-9fe3-fd102227932e","Type":"ContainerStarted","Data":"8a2958363976da55aa16396375b01782691e7c61d6d9de77f1c5f98f3502db9f"}
May 29 02:28:49.018082 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:49.017822    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:28:49.036779 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:28:49.036635    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-w5mh6" podStartSLOduration=67.469035035 podStartE2EDuration="1m10.036618761s" podCreationTimestamp="2026-05-29 02:27:39 +0000 UTC" firstStartedPulling="2026-05-29 02:28:45.762198214 +0000 UTC m=+66.625810723" lastFinishedPulling="2026-05-29 02:28:48.329781936 +0000 UTC m=+69.193394449" observedRunningTime="2026-05-29 02:28:49.035862681 +0000 UTC m=+69.899475213" watchObservedRunningTime="2026-05-29 02:28:49.036618761 +0000 UTC m=+69.900231293"
May 29 02:29:17.398028 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:29:17.397898    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert\") pod \"ingress-canary-fzhb8\" (UID: \"2a4ccab7-f230-42f0-86ba-97f21234b006\") " pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:29:17.398028 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:29:17.397934    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:29:17.398509 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:29:17.398038    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 02:29:17.398509 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:29:17.398047    2553 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 02:29:17.398509 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:29:17.398102    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls podName:21d4bc93-ef47-4e84-b0ea-4c77774456ea nodeName:}" failed. No retries permitted until 2026-05-29 02:30:21.398088454 +0000 UTC m=+162.261700964 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls") pod "dns-default-d4shx" (UID: "21d4bc93-ef47-4e84-b0ea-4c77774456ea") : secret "dns-default-metrics-tls" not found
May 29 02:29:17.398509 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:29:17.398115    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert podName:2a4ccab7-f230-42f0-86ba-97f21234b006 nodeName:}" failed. No retries permitted until 2026-05-29 02:30:21.398109186 +0000 UTC m=+162.261721696 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert") pod "ingress-canary-fzhb8" (UID: "2a4ccab7-f230-42f0-86ba-97f21234b006") : secret "canary-serving-cert" not found
May 29 02:29:20.022565 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:29:20.022536    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-w5mh6"
May 29 02:29:49.413490 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:29:49.413451    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs\") pod \"network-metrics-daemon-7sv8v\" (UID: \"9a900d97-0943-4380-a1e5-85bd64a30ba3\") " pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:29:49.413966 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:29:49.413560    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 29 02:29:49.413966 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:29:49.413618    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs podName:9a900d97-0943-4380-a1e5-85bd64a30ba3 nodeName:}" failed. No retries permitted until 2026-05-29 02:31:51.413601402 +0000 UTC m=+252.277213912 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs") pod "network-metrics-daemon-7sv8v" (UID: "9a900d97-0943-4380-a1e5-85bd64a30ba3") : secret "metrics-daemon-secret" not found
May 29 02:29:50.717253 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:29:50.717223    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w7gt7_d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1/dns-node-resolver/0.log"
May 29 02:29:51.917737 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:29:51.917711    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t8ct4_96b44fa7-1fa7-4811-8513-8d367a40ed40/node-ca/0.log"
May 29 02:30:14.243718 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.243677    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-htf5k"]
May 29 02:30:14.245715 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.245693    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.250073 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.250048    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 29 02:30:14.250212 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.250052    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 29 02:30:14.250336 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.250311    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 29 02:30:14.250773 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.250757    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 29 02:30:14.251049 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.251035    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zxmms\""
May 29 02:30:14.258365 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.258344    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-htf5k"]
May 29 02:30:14.292335 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.292306    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6-data-volume\") pod \"insights-runtime-extractor-htf5k\" (UID: \"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6\") " pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.292441 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.292361    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fg2x\" (UniqueName: \"kubernetes.io/projected/7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6-kube-api-access-4fg2x\") pod \"insights-runtime-extractor-htf5k\" (UID: \"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6\") " pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.292441 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.292381    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6-crio-socket\") pod \"insights-runtime-extractor-htf5k\" (UID: \"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6\") " pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.292441 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.292416    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-htf5k\" (UID: \"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6\") " pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.292543 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.292447    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-htf5k\" (UID: \"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6\") " pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.393211 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.393181    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4fg2x\" (UniqueName: \"kubernetes.io/projected/7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6-kube-api-access-4fg2x\") pod \"insights-runtime-extractor-htf5k\" (UID: \"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6\") " pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.393211 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.393214    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6-crio-socket\") pod \"insights-runtime-extractor-htf5k\" (UID: \"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6\") " pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.393394 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.393236    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-htf5k\" (UID: \"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6\") " pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.393394 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.393320    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6-crio-socket\") pod \"insights-runtime-extractor-htf5k\" (UID: \"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6\") " pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.393394 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.393369    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-htf5k\" (UID: \"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6\") " pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.393480 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.393410    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6-data-volume\") pod \"insights-runtime-extractor-htf5k\" (UID: \"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6\") " pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.393699 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.393670    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6-data-volume\") pod \"insights-runtime-extractor-htf5k\" (UID: \"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6\") " pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.393818 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.393742    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-htf5k\" (UID: \"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6\") " pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.395875 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.395860    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-htf5k\" (UID: \"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6\") " pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.409561 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.409534    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fg2x\" (UniqueName: \"kubernetes.io/projected/7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6-kube-api-access-4fg2x\") pod \"insights-runtime-extractor-htf5k\" (UID: \"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6\") " pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.553720 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.553690    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-htf5k"
May 29 02:30:14.692761 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:14.692730    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-htf5k"]
May 29 02:30:14.695871 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:30:14.695837    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fd7dcc6_dbfa_4c24_a6d8_4557b88316b6.slice/crio-16be07fe7c6a148eb5bfb9cf415c3ffb6108af9dfeee2b00201261f39f73031b WatchSource:0}: Error finding container 16be07fe7c6a148eb5bfb9cf415c3ffb6108af9dfeee2b00201261f39f73031b: Status 404 returned error can't find the container with id 16be07fe7c6a148eb5bfb9cf415c3ffb6108af9dfeee2b00201261f39f73031b
May 29 02:30:15.189765 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:15.189730    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-htf5k" event={"ID":"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6","Type":"ContainerStarted","Data":"c4936d119c48dab90a60f76788e40f35d6a0082217ffa54e1d043918cc5dd542"}
May 29 02:30:15.189929 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:15.189773    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-htf5k" event={"ID":"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6","Type":"ContainerStarted","Data":"16be07fe7c6a148eb5bfb9cf415c3ffb6108af9dfeee2b00201261f39f73031b"}
May 29 02:30:16.193796 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:16.193752    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-htf5k" event={"ID":"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6","Type":"ContainerStarted","Data":"c290643d00edcbe1ff83113c680e104d807190f1f97f7814b85d26eaadfdcb8c"}
May 29 02:30:16.533017 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:30:16.532967    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-fzhb8" podUID="2a4ccab7-f230-42f0-86ba-97f21234b006"
May 29 02:30:16.545785 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:30:16.545749    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-d4shx" podUID="21d4bc93-ef47-4e84-b0ea-4c77774456ea"
May 29 02:30:17.198151 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:17.198116    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:30:17.198151 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:17.198116    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-htf5k" event={"ID":"7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6","Type":"ContainerStarted","Data":"79c67fe6f733debd655bafc491b67cdce01bd3e966cf2f52253fb223e6081566"}
May 29 02:30:17.732505 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:30:17.732470    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-7sv8v" podUID="9a900d97-0943-4380-a1e5-85bd64a30ba3"
May 29 02:30:21.209792 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:21.209752    2553 generic.go:358] "Generic (PLEG): container finished" podID="63b6ddb5-4ab9-41c7-8687-54230ad2de82" containerID="83d85bd29649ed31ee8be6cbd60083cbde12fa197c4190ef2640118399d35c2e" exitCode=255
May 29 02:30:21.210269 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:21.209827    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65674dc458-wggjj" event={"ID":"63b6ddb5-4ab9-41c7-8687-54230ad2de82","Type":"ContainerDied","Data":"83d85bd29649ed31ee8be6cbd60083cbde12fa197c4190ef2640118399d35c2e"}
May 29 02:30:21.210269 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:21.210138    2553 scope.go:117] "RemoveContainer" containerID="83d85bd29649ed31ee8be6cbd60083cbde12fa197c4190ef2640118399d35c2e"
May 29 02:30:21.227626 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:21.227587    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-htf5k" podStartSLOduration=5.359390388 podStartE2EDuration="7.227574998s" podCreationTimestamp="2026-05-29 02:30:14 +0000 UTC" firstStartedPulling="2026-05-29 02:30:14.74059225 +0000 UTC m=+155.604204760" lastFinishedPulling="2026-05-29 02:30:16.608776856 +0000 UTC m=+157.472389370" observedRunningTime="2026-05-29 02:30:17.232232971 +0000 UTC m=+158.095845503" watchObservedRunningTime="2026-05-29 02:30:21.227574998 +0000 UTC m=+162.091187529"
May 29 02:30:21.440184 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:21.440138    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert\") pod \"ingress-canary-fzhb8\" (UID: \"2a4ccab7-f230-42f0-86ba-97f21234b006\") " pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:30:21.440184 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:21.440188    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:30:21.442685 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:21.442631    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/21d4bc93-ef47-4e84-b0ea-4c77774456ea-metrics-tls\") pod \"dns-default-d4shx\" (UID: \"21d4bc93-ef47-4e84-b0ea-4c77774456ea\") " pod="openshift-dns/dns-default-d4shx"
May 29 02:30:21.442805 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:21.442737    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2a4ccab7-f230-42f0-86ba-97f21234b006-cert\") pod \"ingress-canary-fzhb8\" (UID: \"2a4ccab7-f230-42f0-86ba-97f21234b006\") " pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:30:21.701638 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:21.701601    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jrg6b\""
May 29 02:30:21.708996 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:21.708974    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fzhb8"
May 29 02:30:21.829148 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:21.829119    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fzhb8"]
May 29 02:30:21.831880 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:30:21.831850    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a4ccab7_f230_42f0_86ba_97f21234b006.slice/crio-621e745872672b69a5ff680ad49dd397f7231cb945a3beed42ee5bebe92ad575 WatchSource:0}: Error finding container 621e745872672b69a5ff680ad49dd397f7231cb945a3beed42ee5bebe92ad575: Status 404 returned error can't find the container with id 621e745872672b69a5ff680ad49dd397f7231cb945a3beed42ee5bebe92ad575
May 29 02:30:22.214251 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:22.214206    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65674dc458-wggjj" event={"ID":"63b6ddb5-4ab9-41c7-8687-54230ad2de82","Type":"ContainerStarted","Data":"69b1ea4ef757577391adae38151f4b4f7d52cbcff5368f5b8b715ca242b77907"}
May 29 02:30:22.215429 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:22.215399    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fzhb8" event={"ID":"2a4ccab7-f230-42f0-86ba-97f21234b006","Type":"ContainerStarted","Data":"621e745872672b69a5ff680ad49dd397f7231cb945a3beed42ee5bebe92ad575"}
May 29 02:30:24.221722 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:24.221686    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fzhb8" event={"ID":"2a4ccab7-f230-42f0-86ba-97f21234b006","Type":"ContainerStarted","Data":"8209eb303a2a9e5254a2b4b66733b8add639a0abd3474b68c00e8c65689b030e"}
May 29 02:30:24.238538 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:24.238489    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fzhb8" podStartSLOduration=129.793495302 podStartE2EDuration="2m11.238474631s" podCreationTimestamp="2026-05-29 02:28:13 +0000 UTC" firstStartedPulling="2026-05-29 02:30:21.834228196 +0000 UTC m=+162.697840707" lastFinishedPulling="2026-05-29 02:30:23.279207524 +0000 UTC m=+164.142820036" observedRunningTime="2026-05-29 02:30:24.237906602 +0000 UTC m=+165.101519135" watchObservedRunningTime="2026-05-29 02:30:24.238474631 +0000 UTC m=+165.102087163"
May 29 02:30:25.491989 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.491953    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pstps"]
May 29 02:30:25.497069 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.497038    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.500287 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.500266    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jnhk2\""
May 29 02:30:25.500399 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.500286    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 29 02:30:25.500548 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.500530    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 29 02:30:25.500548 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.500541    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 29 02:30:25.501072 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.500600    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 29 02:30:25.501072 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.500674    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 29 02:30:25.501072 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.501044    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 29 02:30:25.573458 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.573424    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/34567433-e80a-4963-a6a9-7da83e82115e-node-exporter-tls\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.573458 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.573458    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/34567433-e80a-4963-a6a9-7da83e82115e-root\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.573637 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.573476    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/34567433-e80a-4963-a6a9-7da83e82115e-node-exporter-textfile\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.573637 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.573498    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34567433-e80a-4963-a6a9-7da83e82115e-metrics-client-ca\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.573637 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.573522    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p7xr\" (UniqueName: \"kubernetes.io/projected/34567433-e80a-4963-a6a9-7da83e82115e-kube-api-access-4p7xr\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.573637 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.573563    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34567433-e80a-4963-a6a9-7da83e82115e-sys\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.573637 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.573583    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/34567433-e80a-4963-a6a9-7da83e82115e-node-exporter-wtmp\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.573637 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.573623    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34567433-e80a-4963-a6a9-7da83e82115e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.573902 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.573716    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/34567433-e80a-4963-a6a9-7da83e82115e-node-exporter-accelerators-collector-config\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.674715 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.674684    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34567433-e80a-4963-a6a9-7da83e82115e-sys\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.674715 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.674715    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/34567433-e80a-4963-a6a9-7da83e82115e-node-exporter-wtmp\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.674889 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.674732    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34567433-e80a-4963-a6a9-7da83e82115e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.674889 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.674787    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34567433-e80a-4963-a6a9-7da83e82115e-sys\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.674889 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.674830    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/34567433-e80a-4963-a6a9-7da83e82115e-node-exporter-accelerators-collector-config\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.674889 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.674875    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/34567433-e80a-4963-a6a9-7da83e82115e-node-exporter-wtmp\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.675022 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.674879    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/34567433-e80a-4963-a6a9-7da83e82115e-node-exporter-tls\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.675022 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.674947    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/34567433-e80a-4963-a6a9-7da83e82115e-root\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.675022 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.674973    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/34567433-e80a-4963-a6a9-7da83e82115e-node-exporter-textfile\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.675022 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.675000    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34567433-e80a-4963-a6a9-7da83e82115e-metrics-client-ca\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.675190 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.675025    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4p7xr\" (UniqueName: \"kubernetes.io/projected/34567433-e80a-4963-a6a9-7da83e82115e-kube-api-access-4p7xr\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.675190 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.675055    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/34567433-e80a-4963-a6a9-7da83e82115e-root\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.675287 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.675265    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/34567433-e80a-4963-a6a9-7da83e82115e-node-exporter-textfile\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.675552 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.675530    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34567433-e80a-4963-a6a9-7da83e82115e-metrics-client-ca\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.675703 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.675644    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/34567433-e80a-4963-a6a9-7da83e82115e-node-exporter-accelerators-collector-config\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.677106 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.677084    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34567433-e80a-4963-a6a9-7da83e82115e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.677279 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.677258    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/34567433-e80a-4963-a6a9-7da83e82115e-node-exporter-tls\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.684344 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.684319    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p7xr\" (UniqueName: \"kubernetes.io/projected/34567433-e80a-4963-a6a9-7da83e82115e-kube-api-access-4p7xr\") pod \"node-exporter-pstps\" (UID: \"34567433-e80a-4963-a6a9-7da83e82115e\") " pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.806053 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:25.806028    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pstps"
May 29 02:30:25.814384 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:30:25.814361    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34567433_e80a_4963_a6a9_7da83e82115e.slice/crio-973e81f0efe0465ea6d64640c0c59e8fd43d4f36f0ded91c0b0b9bbebc6e85d0 WatchSource:0}: Error finding container 973e81f0efe0465ea6d64640c0c59e8fd43d4f36f0ded91c0b0b9bbebc6e85d0: Status 404 returned error can't find the container with id 973e81f0efe0465ea6d64640c0c59e8fd43d4f36f0ded91c0b0b9bbebc6e85d0
May 29 02:30:26.228104 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:26.228024    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pstps" event={"ID":"34567433-e80a-4963-a6a9-7da83e82115e","Type":"ContainerStarted","Data":"973e81f0efe0465ea6d64640c0c59e8fd43d4f36f0ded91c0b0b9bbebc6e85d0"}
May 29 02:30:26.715947 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:26.715923    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-d4shx"
May 29 02:30:26.719273 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:26.719257    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zl2l4\""
May 29 02:30:26.727214 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:26.727196    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-d4shx"
May 29 02:30:26.863784 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:26.863755    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-d4shx"]
May 29 02:30:26.869578 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:30:26.869545    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d4bc93_ef47_4e84_b0ea_4c77774456ea.slice/crio-98f26d4496e6c7304e26335505530cbd2e7e8667364a5194c693ac5defa8dba2 WatchSource:0}: Error finding container 98f26d4496e6c7304e26335505530cbd2e7e8667364a5194c693ac5defa8dba2: Status 404 returned error can't find the container with id 98f26d4496e6c7304e26335505530cbd2e7e8667364a5194c693ac5defa8dba2
May 29 02:30:27.231627 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.231594    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d4shx" event={"ID":"21d4bc93-ef47-4e84-b0ea-4c77774456ea","Type":"ContainerStarted","Data":"98f26d4496e6c7304e26335505530cbd2e7e8667364a5194c693ac5defa8dba2"}
May 29 02:30:27.232861 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.232840    2553 generic.go:358] "Generic (PLEG): container finished" podID="34567433-e80a-4963-a6a9-7da83e82115e" containerID="fdf3c23a17ad968dc28a44912e5aafdfbbb18e9d58a55093e973681fb28f404e" exitCode=0
May 29 02:30:27.232941 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.232870    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pstps" event={"ID":"34567433-e80a-4963-a6a9-7da83e82115e","Type":"ContainerDied","Data":"fdf3c23a17ad968dc28a44912e5aafdfbbb18e9d58a55093e973681fb28f404e"}
May 29 02:30:27.590747 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.590710    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"]
May 29 02:30:27.594115 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.594093    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.613313 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.613286    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
May 29 02:30:27.613313 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.613285    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
May 29 02:30:27.613503 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.613393    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
May 29 02:30:27.613555 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.613529    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-smzl2\""
May 29 02:30:27.614160 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.614141    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-757vh8amr95ps\""
May 29 02:30:27.614243 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.614147    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
May 29 02:30:27.623192 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.623168    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
May 29 02:30:27.691591 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.691475    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-thanos-querier-tls\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.691591 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.691526    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/477f3c90-0808-46bd-a908-4837260e6964-metrics-client-ca\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.691805 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.691616    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-grpc-tls\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.691805 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.691687    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.691805 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.691719    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.691805 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.691762    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tm7b\" (UniqueName: \"kubernetes.io/projected/477f3c90-0808-46bd-a908-4837260e6964-kube-api-access-7tm7b\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.691972 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.691838    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.691972 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.691867    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.710484 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.710454    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"]
May 29 02:30:27.792253 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.792209    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.792713 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.792261    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.792713 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.792316    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-thanos-querier-tls\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.792713 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.792352    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/477f3c90-0808-46bd-a908-4837260e6964-metrics-client-ca\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.792713 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.792381    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-grpc-tls\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.792713 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.792411    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.792713 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.792438    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.792713 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.792476    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7tm7b\" (UniqueName: \"kubernetes.io/projected/477f3c90-0808-46bd-a908-4837260e6964-kube-api-access-7tm7b\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.793469 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.793416    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/477f3c90-0808-46bd-a908-4837260e6964-metrics-client-ca\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.795486 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.795429    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-thanos-querier-tls\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.795817 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.795796    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.795892 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.795862    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.795987 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.795963    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.796096 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.796077    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-grpc-tls\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.796167 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.796139    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/477f3c90-0808-46bd-a908-4837260e6964-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.801768 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.801747    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tm7b\" (UniqueName: \"kubernetes.io/projected/477f3c90-0808-46bd-a908-4837260e6964-kube-api-access-7tm7b\") pod \"thanos-querier-5d646c6b65-gkfgm\" (UID: \"477f3c90-0808-46bd-a908-4837260e6964\") " pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:27.903907 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:27.903823    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:28.237541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:28.237494    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"]
May 29 02:30:28.238306 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:28.238271    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pstps" event={"ID":"34567433-e80a-4963-a6a9-7da83e82115e","Type":"ContainerStarted","Data":"405dacd94c0c8a3475a27f8d25ef00520795bdaf98d1bb8630ad2998e827da2c"}
May 29 02:30:28.238396 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:28.238305    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pstps" event={"ID":"34567433-e80a-4963-a6a9-7da83e82115e","Type":"ContainerStarted","Data":"68e2993522aa64b276266899422b2263a18a20ca298ca487b2007ce5c60af92c"}
May 29 02:30:28.241217 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:30:28.241187    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod477f3c90_0808_46bd_a908_4837260e6964.slice/crio-ab9c8576094fd9e4b3219d64a7842b4402453b4852cb2d3a61502572e6595659 WatchSource:0}: Error finding container ab9c8576094fd9e4b3219d64a7842b4402453b4852cb2d3a61502572e6595659: Status 404 returned error can't find the container with id ab9c8576094fd9e4b3219d64a7842b4402453b4852cb2d3a61502572e6595659
May 29 02:30:29.243353 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.243312    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d4shx" event={"ID":"21d4bc93-ef47-4e84-b0ea-4c77774456ea","Type":"ContainerStarted","Data":"ba04f0f630e20a82184a2d30e4f57b349ee93832564f8cd5f79d3e043d39a26b"}
May 29 02:30:29.243875 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.243361    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d4shx" event={"ID":"21d4bc93-ef47-4e84-b0ea-4c77774456ea","Type":"ContainerStarted","Data":"b047edda332c5e3aece96bbd66494a1ab7251facbd67c4585abf2db6d6f5baec"}
May 29 02:30:29.243875 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.243418    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-d4shx"
May 29 02:30:29.244598 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.244566    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm" event={"ID":"477f3c90-0808-46bd-a908-4837260e6964","Type":"ContainerStarted","Data":"ab9c8576094fd9e4b3219d64a7842b4402453b4852cb2d3a61502572e6595659"}
May 29 02:30:29.269681 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.269621    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-d4shx" podStartSLOduration=134.997031704 podStartE2EDuration="2m16.26960945s" podCreationTimestamp="2026-05-29 02:28:13 +0000 UTC" firstStartedPulling="2026-05-29 02:30:26.871474846 +0000 UTC m=+167.735087356" lastFinishedPulling="2026-05-29 02:30:28.14405257 +0000 UTC m=+169.007665102" observedRunningTime="2026-05-29 02:30:29.268498869 +0000 UTC m=+170.132111400" watchObservedRunningTime="2026-05-29 02:30:29.26960945 +0000 UTC m=+170.133221983"
May 29 02:30:29.269806 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.269777    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pstps" podStartSLOduration=3.4148935160000002 podStartE2EDuration="4.269769951s" podCreationTimestamp="2026-05-29 02:30:25 +0000 UTC" firstStartedPulling="2026-05-29 02:30:25.816478788 +0000 UTC m=+166.680091298" lastFinishedPulling="2026-05-29 02:30:26.671355221 +0000 UTC m=+167.534967733" observedRunningTime="2026-05-29 02:30:28.259233762 +0000 UTC m=+169.122846298" watchObservedRunningTime="2026-05-29 02:30:29.269769951 +0000 UTC m=+170.133382464"
May 29 02:30:29.799343 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.799306    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5c96c6856c-rrhjm"]
May 29 02:30:29.802759 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.802724    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:29.805363 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.805332    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
May 29 02:30:29.805475 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.805412    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
May 29 02:30:29.805554 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.805521    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-xhjkh\""
May 29 02:30:29.805554 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.805550    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 29 02:30:29.805638 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.805599    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
May 29 02:30:29.806049 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.806029    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-f3bv4o10itqme\""
May 29 02:30:29.811576 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.811555    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c96c6856c-rrhjm"]
May 29 02:30:29.911215 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.911167    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7c206e11-3d41-48bb-b0f7-d297670d7a71-secret-metrics-server-client-certs\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:29.911363 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.911229    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nqrv\" (UniqueName: \"kubernetes.io/projected/7c206e11-3d41-48bb-b0f7-d297670d7a71-kube-api-access-7nqrv\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:29.911363 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.911258    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c206e11-3d41-48bb-b0f7-d297670d7a71-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:29.911363 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.911332    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7c206e11-3d41-48bb-b0f7-d297670d7a71-secret-metrics-server-tls\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:29.911493 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.911383    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7c206e11-3d41-48bb-b0f7-d297670d7a71-client-ca-bundle\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:29.911493 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.911440    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7c206e11-3d41-48bb-b0f7-d297670d7a71-audit-log\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:29.911493 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:29.911474    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7c206e11-3d41-48bb-b0f7-d297670d7a71-metrics-server-audit-profiles\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:30.012414 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.012375    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7c206e11-3d41-48bb-b0f7-d297670d7a71-secret-metrics-server-client-certs\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:30.012587 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.012444    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7nqrv\" (UniqueName: \"kubernetes.io/projected/7c206e11-3d41-48bb-b0f7-d297670d7a71-kube-api-access-7nqrv\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:30.012587 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.012477    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c206e11-3d41-48bb-b0f7-d297670d7a71-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:30.012587 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.012508    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7c206e11-3d41-48bb-b0f7-d297670d7a71-secret-metrics-server-tls\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:30.012587 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.012541    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7c206e11-3d41-48bb-b0f7-d297670d7a71-client-ca-bundle\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:30.012587 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.012579    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7c206e11-3d41-48bb-b0f7-d297670d7a71-audit-log\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:30.012900 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.012617    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7c206e11-3d41-48bb-b0f7-d297670d7a71-metrics-server-audit-profiles\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:30.013068 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.013046    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7c206e11-3d41-48bb-b0f7-d297670d7a71-audit-log\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:30.013585 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.013503    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c206e11-3d41-48bb-b0f7-d297670d7a71-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:30.013927 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.013900    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7c206e11-3d41-48bb-b0f7-d297670d7a71-metrics-server-audit-profiles\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:30.015470 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.015422    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7c206e11-3d41-48bb-b0f7-d297670d7a71-client-ca-bundle\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:30.015470 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.015447    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7c206e11-3d41-48bb-b0f7-d297670d7a71-secret-metrics-server-tls\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:30.015596 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.015492    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7c206e11-3d41-48bb-b0f7-d297670d7a71-secret-metrics-server-client-certs\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:30.022398 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.022373    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nqrv\" (UniqueName: \"kubernetes.io/projected/7c206e11-3d41-48bb-b0f7-d297670d7a71-kube-api-access-7nqrv\") pod \"metrics-server-5c96c6856c-rrhjm\" (UID: \"7c206e11-3d41-48bb-b0f7-d297670d7a71\") " pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:30.113707 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.113615    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:30.249265 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.249235    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm" event={"ID":"477f3c90-0808-46bd-a908-4837260e6964","Type":"ContainerStarted","Data":"24ff16b0194123bdc6b1c7f5e1e8109ee5df4d13c82fed9cff5225513bb61f0c"}
May 29 02:30:30.272012 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.271985    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c96c6856c-rrhjm"]
May 29 02:30:30.274441 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:30:30.274416    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c206e11_3d41_48bb_b0f7_d297670d7a71.slice/crio-f3aa1ad305d9682226b531a6f0486c15156b0fc431768d9d2173a540e137ae52 WatchSource:0}: Error finding container f3aa1ad305d9682226b531a6f0486c15156b0fc431768d9d2173a540e137ae52: Status 404 returned error can't find the container with id f3aa1ad305d9682226b531a6f0486c15156b0fc431768d9d2173a540e137ae52
May 29 02:30:30.691442 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.691394    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-76c88799db-94gsb"]
May 29 02:30:30.694677 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.694638    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.697173 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.697148    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
May 29 02:30:30.697281 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.697201    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
May 29 02:30:30.697281 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.697210    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-bq9b5\""
May 29 02:30:30.697281 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.697157    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
May 29 02:30:30.697281 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.697202    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
May 29 02:30:30.697434 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.697151    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
May 29 02:30:30.702513 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.702497    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
May 29 02:30:30.706395 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.706363    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76c88799db-94gsb"]
May 29 02:30:30.718645 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.718607    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4406c398-9433-4a19-a85f-91bbefd9c3b4-telemeter-client-tls\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.718751 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.718678    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4406c398-9433-4a19-a85f-91bbefd9c3b4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.718796 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.718754    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4406c398-9433-4a19-a85f-91bbefd9c3b4-metrics-client-ca\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.718841 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.718801    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwj8d\" (UniqueName: \"kubernetes.io/projected/4406c398-9433-4a19-a85f-91bbefd9c3b4-kube-api-access-jwj8d\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.718841 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.718827    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4406c398-9433-4a19-a85f-91bbefd9c3b4-secret-telemeter-client\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.718915 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.718864    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4406c398-9433-4a19-a85f-91bbefd9c3b4-federate-client-tls\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.718915 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.718883    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4406c398-9433-4a19-a85f-91bbefd9c3b4-serving-certs-ca-bundle\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.718984 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.718917    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4406c398-9433-4a19-a85f-91bbefd9c3b4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.819928 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.819880    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4406c398-9433-4a19-a85f-91bbefd9c3b4-metrics-client-ca\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.820121 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.819937    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jwj8d\" (UniqueName: \"kubernetes.io/projected/4406c398-9433-4a19-a85f-91bbefd9c3b4-kube-api-access-jwj8d\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.820121 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.819973    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4406c398-9433-4a19-a85f-91bbefd9c3b4-secret-telemeter-client\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.820234 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.820134    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4406c398-9433-4a19-a85f-91bbefd9c3b4-federate-client-tls\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.820234 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.820187    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4406c398-9433-4a19-a85f-91bbefd9c3b4-serving-certs-ca-bundle\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.820352 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.820327    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4406c398-9433-4a19-a85f-91bbefd9c3b4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.820417 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.820374    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4406c398-9433-4a19-a85f-91bbefd9c3b4-telemeter-client-tls\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.820545 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.820512    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4406c398-9433-4a19-a85f-91bbefd9c3b4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.820701 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.820681    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4406c398-9433-4a19-a85f-91bbefd9c3b4-metrics-client-ca\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.820895 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.820872    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4406c398-9433-4a19-a85f-91bbefd9c3b4-serving-certs-ca-bundle\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.821369 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.821316    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4406c398-9433-4a19-a85f-91bbefd9c3b4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.823194 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.823154    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4406c398-9433-4a19-a85f-91bbefd9c3b4-telemeter-client-tls\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.823282 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.823251    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4406c398-9433-4a19-a85f-91bbefd9c3b4-federate-client-tls\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.823348 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.823326    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4406c398-9433-4a19-a85f-91bbefd9c3b4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.823348 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.823341    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4406c398-9433-4a19-a85f-91bbefd9c3b4-secret-telemeter-client\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:30.830046 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:30.830020    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwj8d\" (UniqueName: \"kubernetes.io/projected/4406c398-9433-4a19-a85f-91bbefd9c3b4-kube-api-access-jwj8d\") pod \"telemeter-client-76c88799db-94gsb\" (UID: \"4406c398-9433-4a19-a85f-91bbefd9c3b4\") " pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:31.004853 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.004779    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76c88799db-94gsb"
May 29 02:30:31.167015 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.166982    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76c88799db-94gsb"]
May 29 02:30:31.197070 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:30:31.197033    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4406c398_9433_4a19_a85f_91bbefd9c3b4.slice/crio-ebb6a6d34393d3360a856a17897da3bb08da520cf9be4b847120b8ea716caff2 WatchSource:0}: Error finding container ebb6a6d34393d3360a856a17897da3bb08da520cf9be4b847120b8ea716caff2: Status 404 returned error can't find the container with id ebb6a6d34393d3360a856a17897da3bb08da520cf9be4b847120b8ea716caff2
May 29 02:30:31.253000 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.252968    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76c88799db-94gsb" event={"ID":"4406c398-9433-4a19-a85f-91bbefd9c3b4","Type":"ContainerStarted","Data":"ebb6a6d34393d3360a856a17897da3bb08da520cf9be4b847120b8ea716caff2"}
May 29 02:30:31.254128 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.254107    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm" event={"ID":"7c206e11-3d41-48bb-b0f7-d297670d7a71","Type":"ContainerStarted","Data":"f3aa1ad305d9682226b531a6f0486c15156b0fc431768d9d2173a540e137ae52"}
May 29 02:30:31.255882 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.255827    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm" event={"ID":"477f3c90-0808-46bd-a908-4837260e6964","Type":"ContainerStarted","Data":"b66baaf37c8c33752ca71bc04e75492a33627e61a07c4de0768fdafbb26dc509"}
May 29 02:30:31.255882 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.255856    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm" event={"ID":"477f3c90-0808-46bd-a908-4837260e6964","Type":"ContainerStarted","Data":"855f7a7a402ac481803cb8e005a344041c6975874b47322ea33ad35a02b227c0"}
May 29 02:30:31.811261 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.811041    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:30:31.815525 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.814989    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.825574 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.825551    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p2fg4\""
May 29 02:30:31.825574 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.825551    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 29 02:30:31.825760 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.825594    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 29 02:30:31.826310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.826002    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 29 02:30:31.826310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.826139    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 29 02:30:31.826310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.826185    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 29 02:30:31.826310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.826191    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 29 02:30:31.826566 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.826369    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 29 02:30:31.826566 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.826381    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 29 02:30:31.826566 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.826388    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 29 02:30:31.826566 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.826474    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 29 02:30:31.826566 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.826491    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 29 02:30:31.826566 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.826512    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cj428ultau5ej\""
May 29 02:30:31.826566 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.826545    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 29 02:30:31.835753 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.835727    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:30:31.930693 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.930582    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8398ff3a-66ff-4968-906a-87795035c705-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.930693 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.930630    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.930693 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.930673    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8398ff3a-66ff-4968-906a-87795035c705-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.930956 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.930727    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8398ff3a-66ff-4968-906a-87795035c705-config-out\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.930956 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.930808    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.930956 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.930834    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.930956 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.930887    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-web-config\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.930956 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.930938    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.931193 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.930965    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.931193 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.931002    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.931193 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.931026    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.931193 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.931045    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.931193 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.931067    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.931193 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.931086    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xwkp\" (UniqueName: \"kubernetes.io/projected/8398ff3a-66ff-4968-906a-87795035c705-kube-api-access-7xwkp\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.931470 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.931188    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.931470 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.931237    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-config\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.931470 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.931270    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:31.931470 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:31.931308    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.032580 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.032538    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7xwkp\" (UniqueName: \"kubernetes.io/projected/8398ff3a-66ff-4968-906a-87795035c705-kube-api-access-7xwkp\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.032580 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.032580    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.032850 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.032618    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-config\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.032850 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.032643    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.032850 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.032679    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.032850 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.032708    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8398ff3a-66ff-4968-906a-87795035c705-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.032850 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.032729    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.032850 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.032752    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8398ff3a-66ff-4968-906a-87795035c705-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.032850 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.032784    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8398ff3a-66ff-4968-906a-87795035c705-config-out\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.032850 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.032843    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.033238 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.032869    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.033238 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.032896    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-web-config\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.033238 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.032922    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.033238 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.032944    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.033238 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.032972    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.033238 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.032997    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.033238 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.033031    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.033238 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.033056    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.034256 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.033941    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8398ff3a-66ff-4968-906a-87795035c705-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.034256 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.034023    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.034435 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.034346    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.034673 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.034637    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.037063 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.037036    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-config\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.037721 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.037423    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.037721 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.037607    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.037872 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.037820    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.038396 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.038349    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8398ff3a-66ff-4968-906a-87795035c705-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.038499 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.038473    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.039036 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.038986    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.039036 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.039016    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-web-config\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.039179 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.039060    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.039590 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.039567    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.039824 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.039807    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8398ff3a-66ff-4968-906a-87795035c705-config-out\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.040043 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.040017    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.040686 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.040641    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.042356 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.042338    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xwkp\" (UniqueName: \"kubernetes.io/projected/8398ff3a-66ff-4968-906a-87795035c705-kube-api-access-7xwkp\") pod \"prometheus-k8s-0\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.127707 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.127608    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:32.259545 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.259507    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm" event={"ID":"7c206e11-3d41-48bb-b0f7-d297670d7a71","Type":"ContainerStarted","Data":"ced45fdb9690a85c724b653eb5d6b02db2155a487df276cc0bb1e9bdc9ea80c5"}
May 29 02:30:32.262045 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.262021    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm" event={"ID":"477f3c90-0808-46bd-a908-4837260e6964","Type":"ContainerStarted","Data":"55e0dd25e366e7712736e8b14611fd8326d79f574843e3b18e11a85064734fba"}
May 29 02:30:32.262118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.262050    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm" event={"ID":"477f3c90-0808-46bd-a908-4837260e6964","Type":"ContainerStarted","Data":"59cf03af5b924b839403a6534b7103e83e6ba2e1ed1562862308554839850fec"}
May 29 02:30:32.262118 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.262059    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm" event={"ID":"477f3c90-0808-46bd-a908-4837260e6964","Type":"ContainerStarted","Data":"024926b862d461a2f0af003d893a70bf9aa175ec0be630ee7a52ecbba122c803"}
May 29 02:30:32.262214 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.262195    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:32.275524 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.275501    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:30:32.278121 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:30:32.277628    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8398ff3a_66ff_4968_906a_87795035c705.slice/crio-01b3a7761d98c3e5bcfb42d9dc7e3bff82d022fd4c05e9d1eb90f20698e438e6 WatchSource:0}: Error finding container 01b3a7761d98c3e5bcfb42d9dc7e3bff82d022fd4c05e9d1eb90f20698e438e6: Status 404 returned error can't find the container with id 01b3a7761d98c3e5bcfb42d9dc7e3bff82d022fd4c05e9d1eb90f20698e438e6
May 29 02:30:32.279157 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.279065    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm" podStartSLOduration=1.8986347540000001 podStartE2EDuration="3.279048767s" podCreationTimestamp="2026-05-29 02:30:29 +0000 UTC" firstStartedPulling="2026-05-29 02:30:30.27653694 +0000 UTC m=+171.140149450" lastFinishedPulling="2026-05-29 02:30:31.656950948 +0000 UTC m=+172.520563463" observedRunningTime="2026-05-29 02:30:32.278736809 +0000 UTC m=+173.142349342" watchObservedRunningTime="2026-05-29 02:30:32.279048767 +0000 UTC m=+173.142661300"
May 29 02:30:32.302545 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.302494    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm" podStartSLOduration=1.929476632 podStartE2EDuration="5.302474285s" podCreationTimestamp="2026-05-29 02:30:27 +0000 UTC" firstStartedPulling="2026-05-29 02:30:28.243016917 +0000 UTC m=+169.106629427" lastFinishedPulling="2026-05-29 02:30:31.616014565 +0000 UTC m=+172.479627080" observedRunningTime="2026-05-29 02:30:32.301293263 +0000 UTC m=+173.164905795" watchObservedRunningTime="2026-05-29 02:30:32.302474285 +0000 UTC m=+173.166086837"
May 29 02:30:32.715922 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:32.715886    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:30:33.266361 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:33.266315    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76c88799db-94gsb" event={"ID":"4406c398-9433-4a19-a85f-91bbefd9c3b4","Type":"ContainerStarted","Data":"f9f6969e417e77fc897e96eedd397b4781161099c9955e3076bbc82ff56e2e10"}
May 29 02:30:33.267720 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:33.267687    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8398ff3a-66ff-4968-906a-87795035c705","Type":"ContainerStarted","Data":"01b3a7761d98c3e5bcfb42d9dc7e3bff82d022fd4c05e9d1eb90f20698e438e6"}
May 29 02:30:34.272034 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:34.271997    2553 generic.go:358] "Generic (PLEG): container finished" podID="8398ff3a-66ff-4968-906a-87795035c705" containerID="252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5" exitCode=0
May 29 02:30:34.272413 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:34.272077    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8398ff3a-66ff-4968-906a-87795035c705","Type":"ContainerDied","Data":"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5"}
May 29 02:30:34.274014 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:34.273990    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76c88799db-94gsb" event={"ID":"4406c398-9433-4a19-a85f-91bbefd9c3b4","Type":"ContainerStarted","Data":"66775e27cc25eb96ec1dacd3f9b78c0e9313dd953a96cb2e5e93de1b3ee8ca86"}
May 29 02:30:34.274087 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:34.274022    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76c88799db-94gsb" event={"ID":"4406c398-9433-4a19-a85f-91bbefd9c3b4","Type":"ContainerStarted","Data":"7c87f86b3f63f3ec00edfadb51260754e0aaa2961aee945f8f80ea5f348c0779"}
May 29 02:30:34.328466 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:34.328407    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-76c88799db-94gsb" podStartSLOduration=1.8826455960000001 podStartE2EDuration="4.328394039s" podCreationTimestamp="2026-05-29 02:30:30 +0000 UTC" firstStartedPulling="2026-05-29 02:30:31.199260644 +0000 UTC m=+172.062873155" lastFinishedPulling="2026-05-29 02:30:33.645009087 +0000 UTC m=+174.508621598" observedRunningTime="2026-05-29 02:30:34.326097118 +0000 UTC m=+175.189709692" watchObservedRunningTime="2026-05-29 02:30:34.328394039 +0000 UTC m=+175.192006568"
May 29 02:30:37.286717 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:37.286600    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8398ff3a-66ff-4968-906a-87795035c705","Type":"ContainerStarted","Data":"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290"}
May 29 02:30:37.286717 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:37.286637    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8398ff3a-66ff-4968-906a-87795035c705","Type":"ContainerStarted","Data":"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7"}
May 29 02:30:37.286717 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:37.286647    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8398ff3a-66ff-4968-906a-87795035c705","Type":"ContainerStarted","Data":"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220"}
May 29 02:30:37.286717 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:37.286684    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8398ff3a-66ff-4968-906a-87795035c705","Type":"ContainerStarted","Data":"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18"}
May 29 02:30:37.286717 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:37.286692    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8398ff3a-66ff-4968-906a-87795035c705","Type":"ContainerStarted","Data":"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f"}
May 29 02:30:37.286717 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:37.286700    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8398ff3a-66ff-4968-906a-87795035c705","Type":"ContainerStarted","Data":"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d"}
May 29 02:30:37.317663 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:37.317601    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.8296468620000002 podStartE2EDuration="6.317586786s" podCreationTimestamp="2026-05-29 02:30:31 +0000 UTC" firstStartedPulling="2026-05-29 02:30:32.281256894 +0000 UTC m=+173.144869421" lastFinishedPulling="2026-05-29 02:30:36.769196835 +0000 UTC m=+177.632809345" observedRunningTime="2026-05-29 02:30:37.31535712 +0000 UTC m=+178.178969651" watchObservedRunningTime="2026-05-29 02:30:37.317586786 +0000 UTC m=+178.181199317"
May 29 02:30:38.273867 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:38.273837    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5d646c6b65-gkfgm"
May 29 02:30:39.251549 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:39.251519    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-d4shx"
May 29 02:30:42.128139 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:42.128103    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:30:50.113998 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:50.113957    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:50.114382 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:50.114038    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:30:55.253231 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:30:55.253180    2553 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9" podUID="88059012-e845-47ed-96e2-4b4262703155" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 29 02:31:05.252745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:05.252697    2553 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9" podUID="88059012-e845-47ed-96e2-4b4262703155" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 29 02:31:10.119324 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:10.119295    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:31:10.123103 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:10.123075    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5c96c6856c-rrhjm"
May 29 02:31:15.252868 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:15.252832    2553 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9" podUID="88059012-e845-47ed-96e2-4b4262703155" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 29 02:31:15.253314 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:15.252910    2553 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9"
May 29 02:31:15.253493 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:15.253461    2553 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"8600b81e01adbb5685c50675bffbf68271377c3c4ffb8e70e969a043de57cb25"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9" containerMessage="Container service-proxy failed liveness probe, will be restarted"
May 29 02:31:15.253550 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:15.253534    2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9" podUID="88059012-e845-47ed-96e2-4b4262703155" containerName="service-proxy" containerID="cri-o://8600b81e01adbb5685c50675bffbf68271377c3c4ffb8e70e969a043de57cb25" gracePeriod=30
May 29 02:31:15.401558 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:15.401514    2553 generic.go:358] "Generic (PLEG): container finished" podID="88059012-e845-47ed-96e2-4b4262703155" containerID="8600b81e01adbb5685c50675bffbf68271377c3c4ffb8e70e969a043de57cb25" exitCode=2
May 29 02:31:15.401680 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:15.401584    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9" event={"ID":"88059012-e845-47ed-96e2-4b4262703155","Type":"ContainerDied","Data":"8600b81e01adbb5685c50675bffbf68271377c3c4ffb8e70e969a043de57cb25"}
May 29 02:31:16.406603 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:16.406568    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69d8fd48b6-2hkq9" event={"ID":"88059012-e845-47ed-96e2-4b4262703155","Type":"ContainerStarted","Data":"198fedc7b69eeb56620795721ad1f84a45760d117902cca4a62edb1d942dc712"}
May 29 02:31:32.128528 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:32.128471    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:32.149004 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:32.148979    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:32.466838 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:32.466760    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:50.202734 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.202699    2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:31:50.203168 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.203129    2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="prometheus" containerID="cri-o://269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d" gracePeriod=600
May 29 02:31:50.203168 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.203158    2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="config-reloader" containerID="cri-o://172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f" gracePeriod=600
May 29 02:31:50.203376 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.203183    2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290" gracePeriod=600
May 29 02:31:50.203376 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.203132    2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="kube-rbac-proxy" containerID="cri-o://d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7" gracePeriod=600
May 29 02:31:50.203376 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.203159    2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="thanos-sidecar" containerID="cri-o://a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18" gracePeriod=600
May 29 02:31:50.203376 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.203208    2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="kube-rbac-proxy-web" containerID="cri-o://aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220" gracePeriod=600
May 29 02:31:50.438331 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.438309    2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:50.505809 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.505721    2553 generic.go:358] "Generic (PLEG): container finished" podID="8398ff3a-66ff-4968-906a-87795035c705" containerID="d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290" exitCode=0
May 29 02:31:50.505809 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.505758    2553 generic.go:358] "Generic (PLEG): container finished" podID="8398ff3a-66ff-4968-906a-87795035c705" containerID="d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7" exitCode=0
May 29 02:31:50.505809 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.505767    2553 generic.go:358] "Generic (PLEG): container finished" podID="8398ff3a-66ff-4968-906a-87795035c705" containerID="aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220" exitCode=0
May 29 02:31:50.505809 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.505775    2553 generic.go:358] "Generic (PLEG): container finished" podID="8398ff3a-66ff-4968-906a-87795035c705" containerID="a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18" exitCode=0
May 29 02:31:50.505809 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.505783    2553 generic.go:358] "Generic (PLEG): container finished" podID="8398ff3a-66ff-4968-906a-87795035c705" containerID="172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f" exitCode=0
May 29 02:31:50.505809 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.505791    2553 generic.go:358] "Generic (PLEG): container finished" podID="8398ff3a-66ff-4968-906a-87795035c705" containerID="269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d" exitCode=0
May 29 02:31:50.505809 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.505796    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8398ff3a-66ff-4968-906a-87795035c705","Type":"ContainerDied","Data":"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290"}
May 29 02:31:50.506262 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.505838    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8398ff3a-66ff-4968-906a-87795035c705","Type":"ContainerDied","Data":"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7"}
May 29 02:31:50.506262 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.505849    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8398ff3a-66ff-4968-906a-87795035c705","Type":"ContainerDied","Data":"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220"}
May 29 02:31:50.506262 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.505859    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8398ff3a-66ff-4968-906a-87795035c705","Type":"ContainerDied","Data":"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18"}
May 29 02:31:50.506262 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.505868    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8398ff3a-66ff-4968-906a-87795035c705","Type":"ContainerDied","Data":"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f"}
May 29 02:31:50.506262 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.505877    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8398ff3a-66ff-4968-906a-87795035c705","Type":"ContainerDied","Data":"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d"}
May 29 02:31:50.506262 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.505886    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8398ff3a-66ff-4968-906a-87795035c705","Type":"ContainerDied","Data":"01b3a7761d98c3e5bcfb42d9dc7e3bff82d022fd4c05e9d1eb90f20698e438e6"}
May 29 02:31:50.506262 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.505876    2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:50.506262 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.505904    2553 scope.go:117] "RemoveContainer" containerID="d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290"
May 29 02:31:50.513219 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.513197    2553 scope.go:117] "RemoveContainer" containerID="d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7"
May 29 02:31:50.519930 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.519911    2553 scope.go:117] "RemoveContainer" containerID="aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220"
May 29 02:31:50.526950 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.526927    2553 scope.go:117] "RemoveContainer" containerID="a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18"
May 29 02:31:50.533498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.533483    2553 scope.go:117] "RemoveContainer" containerID="172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f"
May 29 02:31:50.539424 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.539409    2553 scope.go:117] "RemoveContainer" containerID="269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d"
May 29 02:31:50.545821 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.545804    2553 scope.go:117] "RemoveContainer" containerID="252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5"
May 29 02:31:50.551668 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.551638    2553 scope.go:117] "RemoveContainer" containerID="d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290"
May 29 02:31:50.551934 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:31:50.551916    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290\": container with ID starting with d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290 not found: ID does not exist" containerID="d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290"
May 29 02:31:50.551977 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.551942    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290"} err="failed to get container status \"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290\": rpc error: code = NotFound desc = could not find container \"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290\": container with ID starting with d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290 not found: ID does not exist"
May 29 02:31:50.551977 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.551960    2553 scope.go:117] "RemoveContainer" containerID="d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7"
May 29 02:31:50.552141 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:31:50.552128    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7\": container with ID starting with d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7 not found: ID does not exist" containerID="d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7"
May 29 02:31:50.552180 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.552144    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7"} err="failed to get container status \"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7\": rpc error: code = NotFound desc = could not find container \"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7\": container with ID starting with d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7 not found: ID does not exist"
May 29 02:31:50.552180 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.552155    2553 scope.go:117] "RemoveContainer" containerID="aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220"
May 29 02:31:50.552382 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:31:50.552365    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220\": container with ID starting with aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220 not found: ID does not exist" containerID="aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220"
May 29 02:31:50.552424 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.552387    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220"} err="failed to get container status \"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220\": rpc error: code = NotFound desc = could not find container \"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220\": container with ID starting with aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220 not found: ID does not exist"
May 29 02:31:50.552424 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.552400    2553 scope.go:117] "RemoveContainer" containerID="a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18"
May 29 02:31:50.552633 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:31:50.552614    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18\": container with ID starting with a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18 not found: ID does not exist" containerID="a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18"
May 29 02:31:50.552713 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.552666    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18"} err="failed to get container status \"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18\": rpc error: code = NotFound desc = could not find container \"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18\": container with ID starting with a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18 not found: ID does not exist"
May 29 02:31:50.552713 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.552688    2553 scope.go:117] "RemoveContainer" containerID="172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f"
May 29 02:31:50.552924 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:31:50.552908    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f\": container with ID starting with 172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f not found: ID does not exist" containerID="172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f"
May 29 02:31:50.552986 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.552933    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f"} err="failed to get container status \"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f\": rpc error: code = NotFound desc = could not find container \"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f\": container with ID starting with 172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f not found: ID does not exist"
May 29 02:31:50.552986 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.552957    2553 scope.go:117] "RemoveContainer" containerID="269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d"
May 29 02:31:50.553184 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:31:50.553169    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d\": container with ID starting with 269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d not found: ID does not exist" containerID="269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d"
May 29 02:31:50.553217 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.553187    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d"} err="failed to get container status \"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d\": rpc error: code = NotFound desc = could not find container \"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d\": container with ID starting with 269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d not found: ID does not exist"
May 29 02:31:50.553217 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.553201    2553 scope.go:117] "RemoveContainer" containerID="252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5"
May 29 02:31:50.553422 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:31:50.553407    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5\": container with ID starting with 252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5 not found: ID does not exist" containerID="252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5"
May 29 02:31:50.553461 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.553426    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5"} err="failed to get container status \"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5\": rpc error: code = NotFound desc = could not find container \"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5\": container with ID starting with 252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5 not found: ID does not exist"
May 29 02:31:50.553461 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.553443    2553 scope.go:117] "RemoveContainer" containerID="d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290"
May 29 02:31:50.553670 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.553637    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290"} err="failed to get container status \"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290\": rpc error: code = NotFound desc = could not find container \"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290\": container with ID starting with d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290 not found: ID does not exist"
May 29 02:31:50.553739 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.553671    2553 scope.go:117] "RemoveContainer" containerID="d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7"
May 29 02:31:50.553883 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.553866    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7"} err="failed to get container status \"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7\": rpc error: code = NotFound desc = could not find container \"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7\": container with ID starting with d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7 not found: ID does not exist"
May 29 02:31:50.553930 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.553884    2553 scope.go:117] "RemoveContainer" containerID="aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220"
May 29 02:31:50.554067 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.554051    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220"} err="failed to get container status \"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220\": rpc error: code = NotFound desc = could not find container \"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220\": container with ID starting with aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220 not found: ID does not exist"
May 29 02:31:50.554108 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.554067    2553 scope.go:117] "RemoveContainer" containerID="a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18"
May 29 02:31:50.554217 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.554198    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18"} err="failed to get container status \"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18\": rpc error: code = NotFound desc = could not find container \"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18\": container with ID starting with a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18 not found: ID does not exist"
May 29 02:31:50.554266 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.554219    2553 scope.go:117] "RemoveContainer" containerID="172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f"
May 29 02:31:50.554440 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.554421    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f"} err="failed to get container status \"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f\": rpc error: code = NotFound desc = could not find container \"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f\": container with ID starting with 172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f not found: ID does not exist"
May 29 02:31:50.554509 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.554441    2553 scope.go:117] "RemoveContainer" containerID="269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d"
May 29 02:31:50.554634 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.554618    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d"} err="failed to get container status \"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d\": rpc error: code = NotFound desc = could not find container \"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d\": container with ID starting with 269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d not found: ID does not exist"
May 29 02:31:50.554696 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.554635    2553 scope.go:117] "RemoveContainer" containerID="252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5"
May 29 02:31:50.554848 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.554831    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5"} err="failed to get container status \"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5\": rpc error: code = NotFound desc = could not find container \"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5\": container with ID starting with 252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5 not found: ID does not exist"
May 29 02:31:50.554894 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.554848    2553 scope.go:117] "RemoveContainer" containerID="d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290"
May 29 02:31:50.555015 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.555000    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290"} err="failed to get container status \"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290\": rpc error: code = NotFound desc = could not find container \"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290\": container with ID starting with d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290 not found: ID does not exist"
May 29 02:31:50.555051 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.555015    2553 scope.go:117] "RemoveContainer" containerID="d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7"
May 29 02:31:50.555169 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.555153    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7"} err="failed to get container status \"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7\": rpc error: code = NotFound desc = could not find container \"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7\": container with ID starting with d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7 not found: ID does not exist"
May 29 02:31:50.555208 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.555169    2553 scope.go:117] "RemoveContainer" containerID="aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220"
May 29 02:31:50.555327 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.555313    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220"} err="failed to get container status \"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220\": rpc error: code = NotFound desc = could not find container \"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220\": container with ID starting with aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220 not found: ID does not exist"
May 29 02:31:50.555369 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.555327    2553 scope.go:117] "RemoveContainer" containerID="a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18"
May 29 02:31:50.555475 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.555459    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18"} err="failed to get container status \"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18\": rpc error: code = NotFound desc = could not find container \"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18\": container with ID starting with a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18 not found: ID does not exist"
May 29 02:31:50.555509 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.555477    2553 scope.go:117] "RemoveContainer" containerID="172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f"
May 29 02:31:50.555702 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.555687    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f"} err="failed to get container status \"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f\": rpc error: code = NotFound desc = could not find container \"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f\": container with ID starting with 172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f not found: ID does not exist"
May 29 02:31:50.555702 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.555702    2553 scope.go:117] "RemoveContainer" containerID="269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d"
May 29 02:31:50.555904 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.555886    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d"} err="failed to get container status \"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d\": rpc error: code = NotFound desc = could not find container \"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d\": container with ID starting with 269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d not found: ID does not exist"
May 29 02:31:50.555953 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.555904    2553 scope.go:117] "RemoveContainer" containerID="252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5"
May 29 02:31:50.556070 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.556054    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5"} err="failed to get container status \"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5\": rpc error: code = NotFound desc = could not find container \"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5\": container with ID starting with 252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5 not found: ID does not exist"
May 29 02:31:50.556070 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.556069    2553 scope.go:117] "RemoveContainer" containerID="d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290"
May 29 02:31:50.556229 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.556215    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290"} err="failed to get container status \"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290\": rpc error: code = NotFound desc = could not find container \"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290\": container with ID starting with d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290 not found: ID does not exist"
May 29 02:31:50.556229 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.556229    2553 scope.go:117] "RemoveContainer" containerID="d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7"
May 29 02:31:50.556439 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.556422    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7"} err="failed to get container status \"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7\": rpc error: code = NotFound desc = could not find container \"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7\": container with ID starting with d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7 not found: ID does not exist"
May 29 02:31:50.556487 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.556440    2553 scope.go:117] "RemoveContainer" containerID="aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220"
May 29 02:31:50.556619 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.556600    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220"} err="failed to get container status \"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220\": rpc error: code = NotFound desc = could not find container \"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220\": container with ID starting with aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220 not found: ID does not exist"
May 29 02:31:50.556695 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.556620    2553 scope.go:117] "RemoveContainer" containerID="a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18"
May 29 02:31:50.556815 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.556797    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18"} err="failed to get container status \"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18\": rpc error: code = NotFound desc = could not find container \"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18\": container with ID starting with a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18 not found: ID does not exist"
May 29 02:31:50.556858 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.556817    2553 scope.go:117] "RemoveContainer" containerID="172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f"
May 29 02:31:50.556978 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.556959    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f"} err="failed to get container status \"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f\": rpc error: code = NotFound desc = could not find container \"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f\": container with ID starting with 172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f not found: ID does not exist"
May 29 02:31:50.557013 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.556978    2553 scope.go:117] "RemoveContainer" containerID="269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d"
May 29 02:31:50.557131 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.557116    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d"} err="failed to get container status \"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d\": rpc error: code = NotFound desc = could not find container \"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d\": container with ID starting with 269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d not found: ID does not exist"
May 29 02:31:50.557175 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.557134    2553 scope.go:117] "RemoveContainer" containerID="252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5"
May 29 02:31:50.557330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.557313    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5"} err="failed to get container status \"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5\": rpc error: code = NotFound desc = could not find container \"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5\": container with ID starting with 252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5 not found: ID does not exist"
May 29 02:31:50.557330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.557329    2553 scope.go:117] "RemoveContainer" containerID="d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290"
May 29 02:31:50.557529 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.557510    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290"} err="failed to get container status \"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290\": rpc error: code = NotFound desc = could not find container \"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290\": container with ID starting with d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290 not found: ID does not exist"
May 29 02:31:50.557570 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.557530    2553 scope.go:117] "RemoveContainer" containerID="d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7"
May 29 02:31:50.557731 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.557716    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7"} err="failed to get container status \"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7\": rpc error: code = NotFound desc = could not find container \"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7\": container with ID starting with d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7 not found: ID does not exist"
May 29 02:31:50.557776 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.557731    2553 scope.go:117] "RemoveContainer" containerID="aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220"
May 29 02:31:50.557889 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.557873    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220"} err="failed to get container status \"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220\": rpc error: code = NotFound desc = could not find container \"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220\": container with ID starting with aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220 not found: ID does not exist"
May 29 02:31:50.557928 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.557889    2553 scope.go:117] "RemoveContainer" containerID="a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18"
May 29 02:31:50.558074 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.558058    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18"} err="failed to get container status \"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18\": rpc error: code = NotFound desc = could not find container \"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18\": container with ID starting with a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18 not found: ID does not exist"
May 29 02:31:50.558115 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.558076    2553 scope.go:117] "RemoveContainer" containerID="172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f"
May 29 02:31:50.558230 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.558212    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f"} err="failed to get container status \"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f\": rpc error: code = NotFound desc = could not find container \"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f\": container with ID starting with 172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f not found: ID does not exist"
May 29 02:31:50.558269 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.558232    2553 scope.go:117] "RemoveContainer" containerID="269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d"
May 29 02:31:50.558388 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.558374    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d"} err="failed to get container status \"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d\": rpc error: code = NotFound desc = could not find container \"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d\": container with ID starting with 269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d not found: ID does not exist"
May 29 02:31:50.558433 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.558389    2553 scope.go:117] "RemoveContainer" containerID="252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5"
May 29 02:31:50.558577 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.558562    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5"} err="failed to get container status \"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5\": rpc error: code = NotFound desc = could not find container \"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5\": container with ID starting with 252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5 not found: ID does not exist"
May 29 02:31:50.558617 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.558579    2553 scope.go:117] "RemoveContainer" containerID="d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290"
May 29 02:31:50.558796 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.558777    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290"} err="failed to get container status \"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290\": rpc error: code = NotFound desc = could not find container \"d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290\": container with ID starting with d3d9c8c54acfe1fe701c02dd45b83ec3431c4df72af8a124f28403be80b88290 not found: ID does not exist"
May 29 02:31:50.558851 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.558796    2553 scope.go:117] "RemoveContainer" containerID="d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7"
May 29 02:31:50.558968 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.558953    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7"} err="failed to get container status \"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7\": rpc error: code = NotFound desc = could not find container \"d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7\": container with ID starting with d66de7e8372459a132c5fa7f49844b6510b487e159e35c4b49c51f2b2e16f4c7 not found: ID does not exist"
May 29 02:31:50.559009 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.558969    2553 scope.go:117] "RemoveContainer" containerID="aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220"
May 29 02:31:50.559158 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.559136    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220"} err="failed to get container status \"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220\": rpc error: code = NotFound desc = could not find container \"aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220\": container with ID starting with aaaa7cb8f43c1eefe06f05817495821692fcb2b0ff1634d433a46f868bcd2220 not found: ID does not exist"
May 29 02:31:50.559225 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.559160    2553 scope.go:117] "RemoveContainer" containerID="a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18"
May 29 02:31:50.559345 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.559332    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18"} err="failed to get container status \"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18\": rpc error: code = NotFound desc = could not find container \"a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18\": container with ID starting with a989435cecf9854eff70422ddde5b0b28e993ceda60d1dc51495f69913ccee18 not found: ID does not exist"
May 29 02:31:50.559392 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.559345    2553 scope.go:117] "RemoveContainer" containerID="172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f"
May 29 02:31:50.559508 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.559491    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f"} err="failed to get container status \"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f\": rpc error: code = NotFound desc = could not find container \"172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f\": container with ID starting with 172de07d48f6a4d670b78440e168f8dee3133a719e58618c93fe6767b4eced3f not found: ID does not exist"
May 29 02:31:50.559571 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.559509    2553 scope.go:117] "RemoveContainer" containerID="269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d"
May 29 02:31:50.559702 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.559688    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d"} err="failed to get container status \"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d\": rpc error: code = NotFound desc = could not find container \"269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d\": container with ID starting with 269bb74758a5664318e28e7cbdfbf550b49c348fb8d15bf83c26156bfa5f327d not found: ID does not exist"
May 29 02:31:50.559753 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.559703    2553 scope.go:117] "RemoveContainer" containerID="252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5"
May 29 02:31:50.559864 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.559850    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5"} err="failed to get container status \"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5\": rpc error: code = NotFound desc = could not find container \"252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5\": container with ID starting with 252019aae4e44dac899133beb7b0b584fb1adbf27ce9bcab4c152cd9dc1f8bd5 not found: ID does not exist"
May 29 02:31:50.603277 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603254    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8398ff3a-66ff-4968-906a-87795035c705-prometheus-k8s-db\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603350 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603287    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-kubelet-serving-ca-bundle\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603350 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603321    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-metrics-client-ca\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603350 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603343    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-metrics-client-certs\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603504 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603362    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8398ff3a-66ff-4968-906a-87795035c705-config-out\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603504 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603377    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-prometheus-k8s-rulefiles-0\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603504 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603402    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xwkp\" (UniqueName: \"kubernetes.io/projected/8398ff3a-66ff-4968-906a-87795035c705-kube-api-access-7xwkp\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603504 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603430    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-serving-certs-ca-bundle\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603504 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603460    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-thanos-prometheus-http-client-file\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603504 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603489    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-tls\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603817 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603519    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-prometheus-trusted-ca-bundle\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603817 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603555    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603817 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603591    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-kube-rbac-proxy\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603817 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603618    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603817 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603673    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-grpc-tls\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603817 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603720    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8398ff3a-66ff-4968-906a-87795035c705-tls-assets\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603817 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603744    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-web-config\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.603817 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603781    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-config\") pod \"8398ff3a-66ff-4968-906a-87795035c705\" (UID: \"8398ff3a-66ff-4968-906a-87795035c705\") "
May 29 02:31:50.604187 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603828    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 02:31:50.604187 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.603829    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 02:31:50.604187 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.604023    2553 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-metrics-client-ca\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.604187 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.604042    2553 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.604510 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.604468    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8398ff3a-66ff-4968-906a-87795035c705-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 02:31:50.605405 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.605160    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 02:31:50.605405 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.605370    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 02:31:50.606141 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.606113    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 02:31:50.606378 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.606326    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8398ff3a-66ff-4968-906a-87795035c705-config-out" (OuterVolumeSpecName: "config-out") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 02:31:50.606507 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.606485    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8398ff3a-66ff-4968-906a-87795035c705-kube-api-access-7xwkp" (OuterVolumeSpecName: "kube-api-access-7xwkp") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "kube-api-access-7xwkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 02:31:50.606696 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.606639    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:31:50.607100 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.607052    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:31:50.607416 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.607390    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8398ff3a-66ff-4968-906a-87795035c705-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 02:31:50.607511 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.607479    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:31:50.608159 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.608122    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:31:50.608378 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.608360    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:31:50.608561 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.608548    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:31:50.608977 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.608952    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-config" (OuterVolumeSpecName: "config") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:31:50.609274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.609256    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:31:50.618038 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.618019    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-web-config" (OuterVolumeSpecName: "web-config") pod "8398ff3a-66ff-4968-906a-87795035c705" (UID: "8398ff3a-66ff-4968-906a-87795035c705"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:31:50.705036 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.704994    2553 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8398ff3a-66ff-4968-906a-87795035c705-tls-assets\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.705036 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.705033    2553 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-web-config\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.705036 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.705042    2553 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-config\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.705036 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.705050    2553 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8398ff3a-66ff-4968-906a-87795035c705-prometheus-k8s-db\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.705257 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.705061    2553 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-metrics-client-certs\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.705257 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.705072    2553 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8398ff3a-66ff-4968-906a-87795035c705-config-out\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.705257 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.705080    2553 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.705257 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.705090    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xwkp\" (UniqueName: \"kubernetes.io/projected/8398ff3a-66ff-4968-906a-87795035c705-kube-api-access-7xwkp\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.705257 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.705099    2553 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.705257 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.705108    2553 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.705257 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.705117    2553 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.705257 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.705125    2553 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8398ff3a-66ff-4968-906a-87795035c705-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.705257 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.705134    2553 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.705257 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.705142    2553 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-kube-rbac-proxy\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.705257 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.705151    2553 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.705257 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.705160    2553 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8398ff3a-66ff-4968-906a-87795035c705-secret-grpc-tls\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:31:50.833534 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.833504    2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:31:50.838409 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.838387    2553 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:31:50.878725 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.878699    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:31:50.878962 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.878950    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="init-config-reloader"
May 29 02:31:50.879001 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.878964    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="init-config-reloader"
May 29 02:31:50.879001 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.878972    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="prometheus"
May 29 02:31:50.879001 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.878979    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="prometheus"
May 29 02:31:50.879001 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.878986    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="config-reloader"
May 29 02:31:50.879001 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.878991    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="config-reloader"
May 29 02:31:50.879001 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.878998    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="thanos-sidecar"
May 29 02:31:50.879001 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.879003    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="thanos-sidecar"
May 29 02:31:50.879218 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.879017    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="kube-rbac-proxy-web"
May 29 02:31:50.879218 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.879022    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="kube-rbac-proxy-web"
May 29 02:31:50.879218 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.879028    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="kube-rbac-proxy"
May 29 02:31:50.879218 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.879033    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="kube-rbac-proxy"
May 29 02:31:50.879218 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.879040    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="kube-rbac-proxy-thanos"
May 29 02:31:50.879218 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.879045    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="kube-rbac-proxy-thanos"
May 29 02:31:50.879218 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.879083    2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="thanos-sidecar"
May 29 02:31:50.879218 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.879089    2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="kube-rbac-proxy-thanos"
May 29 02:31:50.879218 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.879096    2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="config-reloader"
May 29 02:31:50.879218 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.879103    2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="kube-rbac-proxy"
May 29 02:31:50.879218 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.879109    2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="kube-rbac-proxy-web"
May 29 02:31:50.879218 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.879116    2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="8398ff3a-66ff-4968-906a-87795035c705" containerName="prometheus"
May 29 02:31:50.883115 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.883099    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:50.887955 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.887930    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 29 02:31:50.888058 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.887959    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 29 02:31:50.888058 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.887935    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 29 02:31:50.888058 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.887930    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 29 02:31:50.888204 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.888077    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 29 02:31:50.888312 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.888282    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cj428ultau5ej\""
May 29 02:31:50.888397 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.888344    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 29 02:31:50.888397 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.888349    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 29 02:31:50.888564 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.888392    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 29 02:31:50.889330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.889311    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 29 02:31:50.889330 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.889334    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 29 02:31:50.889462 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.889312    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p2fg4\""
May 29 02:31:50.894092 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.894075    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 29 02:31:50.898204 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.898187    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 29 02:31:50.905236 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:50.905216    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:31:51.006910 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.006865    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.006910 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.006909    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.006910 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.006928    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-config-out\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.007164 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.006975    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.007164 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.006992    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.007164 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.007012    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.007164 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.007029    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.007164 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.007048    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-web-config\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.007164 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.007071    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.007164 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.007089    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.007164 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.007103    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vfgb\" (UniqueName: \"kubernetes.io/projected/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-kube-api-access-6vfgb\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.007164 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.007137    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.007497 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.007173    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.007497 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.007193    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-config\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.007497 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.007208    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.007497 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.007229    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.007497 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.007244    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.007497 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.007293    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.108498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.108402    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-config\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.108498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.108449    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.108498 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.108469    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.108783 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.108580    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.108783 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.108612    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.108783 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.108676    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.108783 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.108700    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.108783 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.108724    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-config-out\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.108783 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.108758    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.108783 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.108780    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.109102 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.108818    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.109102 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.108847    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.109102 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.108881    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-web-config\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.109102 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.108929    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.109102 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.108969    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.109102 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.108996    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6vfgb\" (UniqueName: \"kubernetes.io/projected/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-kube-api-access-6vfgb\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.109102 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.109024    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.109102 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.109071    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.109632 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.109279    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.109783 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.109761    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.109986 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.109962    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.110094 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.110072    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.112027 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.111720    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-config\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.112027 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.111879    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.112027 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.111924    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.112235 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.112096    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.112408 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.112385    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.112477 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.112410    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.112787 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.112757    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.112924 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.112904    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-web-config\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.113044 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.113026    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.114715 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.114696    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.114787 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.114738    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-config-out\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.114787 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.114779    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.115285 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.115268    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.119962 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.119941    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vfgb\" (UniqueName: \"kubernetes.io/projected/27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b-kube-api-access-6vfgb\") pod \"prometheus-k8s-0\" (UID: \"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.193036 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.192992    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:31:51.329358 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.329334    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 02:31:51.331409 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:31:51.331380    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27f4fbf0_b29d_4bd7_83b9_78f9bd962a3b.slice/crio-b17de61a90ff09e487262c468ec702e55afc97718089672c8ef3f1a83e997b62 WatchSource:0}: Error finding container b17de61a90ff09e487262c468ec702e55afc97718089672c8ef3f1a83e997b62: Status 404 returned error can't find the container with id b17de61a90ff09e487262c468ec702e55afc97718089672c8ef3f1a83e997b62
May 29 02:31:51.510808 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.510778    2553 generic.go:358] "Generic (PLEG): container finished" podID="27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b" containerID="75618b1805729732c0d20a45d376da2871da8da266c6e188e9aec1cf169f96dc" exitCode=0
May 29 02:31:51.510938 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.510847    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b","Type":"ContainerDied","Data":"75618b1805729732c0d20a45d376da2871da8da266c6e188e9aec1cf169f96dc"}
May 29 02:31:51.510938 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.510866    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b","Type":"ContainerStarted","Data":"b17de61a90ff09e487262c468ec702e55afc97718089672c8ef3f1a83e997b62"}
May 29 02:31:51.512282 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.512264    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs\") pod \"network-metrics-daemon-7sv8v\" (UID: \"9a900d97-0943-4380-a1e5-85bd64a30ba3\") " pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:31:51.514395 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.514371    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a900d97-0943-4380-a1e5-85bd64a30ba3-metrics-certs\") pod \"network-metrics-daemon-7sv8v\" (UID: \"9a900d97-0943-4380-a1e5-85bd64a30ba3\") " pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:31:51.619263 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.619239    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t2rwn\""
May 29 02:31:51.627103 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.627084    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7sv8v"
May 29 02:31:51.734078 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.734036    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8398ff3a-66ff-4968-906a-87795035c705" path="/var/lib/kubelet/pods/8398ff3a-66ff-4968-906a-87795035c705/volumes"
May 29 02:31:51.820408 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:51.820326    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7sv8v"]
May 29 02:31:51.822242 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:31:51.822214    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a900d97_0943_4380_a1e5_85bd64a30ba3.slice/crio-f62224ec40484f16581b38c8bbd7f75be6800439b0f0b81e16efe91ee25c8687 WatchSource:0}: Error finding container f62224ec40484f16581b38c8bbd7f75be6800439b0f0b81e16efe91ee25c8687: Status 404 returned error can't find the container with id f62224ec40484f16581b38c8bbd7f75be6800439b0f0b81e16efe91ee25c8687
May 29 02:31:52.514993 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:52.514958    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7sv8v" event={"ID":"9a900d97-0943-4380-a1e5-85bd64a30ba3","Type":"ContainerStarted","Data":"f62224ec40484f16581b38c8bbd7f75be6800439b0f0b81e16efe91ee25c8687"}
May 29 02:31:52.518263 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:52.518228    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b","Type":"ContainerStarted","Data":"7f0f4081c2be2e17b606bb1a149289fd4157cafdb6628bc1cbf0ca87ec4353d0"}
May 29 02:31:52.518390 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:52.518267    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b","Type":"ContainerStarted","Data":"b5916a39c3762efbf072e0eaca67349c2791c844619a36b218fc53ce50294e70"}
May 29 02:31:52.518390 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:52.518277    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b","Type":"ContainerStarted","Data":"0e80186d1539c69a84c51a7413ca51abec0fa9c3254977df21158f44b9fb83d0"}
May 29 02:31:52.518390 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:52.518285    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b","Type":"ContainerStarted","Data":"3814fbc5a350082b9463116e82b41afd56baaa08ae4744ff4583d05d68976b53"}
May 29 02:31:52.518390 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:52.518293    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b","Type":"ContainerStarted","Data":"3f7d9b5de9d92baaf05a308742d6062ad5707914d4c6fd2fed5182aad5d090c6"}
May 29 02:31:52.518390 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:52.518301    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b","Type":"ContainerStarted","Data":"2f4e3738eb3f9cdce239f35903e84e4c593463284136b85db61d1ef7fdf01c80"}
May 29 02:31:52.552963 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:52.552891    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.552874625 podStartE2EDuration="2.552874625s" podCreationTimestamp="2026-05-29 02:31:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 02:31:52.550626492 +0000 UTC m=+253.414239037" watchObservedRunningTime="2026-05-29 02:31:52.552874625 +0000 UTC m=+253.416487160"
May 29 02:31:53.522971 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:53.522937    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7sv8v" event={"ID":"9a900d97-0943-4380-a1e5-85bd64a30ba3","Type":"ContainerStarted","Data":"bbb17ce8bdbaee766bc019532fb30c9509b13abb792d4be81885bc870fbab288"}
May 29 02:31:53.522971 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:53.522974    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7sv8v" event={"ID":"9a900d97-0943-4380-a1e5-85bd64a30ba3","Type":"ContainerStarted","Data":"cc3b33a2b519d850e1ca8b2384501d32c0a9557e6cfe3722108cae7f96b3c213"}
May 29 02:31:53.540757 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:53.540706    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7sv8v" podStartSLOduration=253.53117618 podStartE2EDuration="4m14.540690536s" podCreationTimestamp="2026-05-29 02:27:39 +0000 UTC" firstStartedPulling="2026-05-29 02:31:51.824028521 +0000 UTC m=+252.687641031" lastFinishedPulling="2026-05-29 02:31:52.833542876 +0000 UTC m=+253.697155387" observedRunningTime="2026-05-29 02:31:53.538789322 +0000 UTC m=+254.402401851" watchObservedRunningTime="2026-05-29 02:31:53.540690536 +0000 UTC m=+254.404303074"
May 29 02:31:56.194057 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:31:56.194024    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:32:21.651371 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:21.651326    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-fbhfk"]
May 29 02:32:21.654345 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:21.654320    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbhfk"
May 29 02:32:21.656409 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:21.656389    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 29 02:32:21.660266 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:21.660241    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fbhfk"]
May 29 02:32:21.745604 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:21.745569    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6efc3fe2-975d-4036-b764-dfa0eed1c83c-original-pull-secret\") pod \"global-pull-secret-syncer-fbhfk\" (UID: \"6efc3fe2-975d-4036-b764-dfa0eed1c83c\") " pod="kube-system/global-pull-secret-syncer-fbhfk"
May 29 02:32:21.745795 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:21.745629    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6efc3fe2-975d-4036-b764-dfa0eed1c83c-kubelet-config\") pod \"global-pull-secret-syncer-fbhfk\" (UID: \"6efc3fe2-975d-4036-b764-dfa0eed1c83c\") " pod="kube-system/global-pull-secret-syncer-fbhfk"
May 29 02:32:21.745795 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:21.745677    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6efc3fe2-975d-4036-b764-dfa0eed1c83c-dbus\") pod \"global-pull-secret-syncer-fbhfk\" (UID: \"6efc3fe2-975d-4036-b764-dfa0eed1c83c\") " pod="kube-system/global-pull-secret-syncer-fbhfk"
May 29 02:32:21.846159 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:21.846114    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6efc3fe2-975d-4036-b764-dfa0eed1c83c-kubelet-config\") pod \"global-pull-secret-syncer-fbhfk\" (UID: \"6efc3fe2-975d-4036-b764-dfa0eed1c83c\") " pod="kube-system/global-pull-secret-syncer-fbhfk"
May 29 02:32:21.846358 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:21.846173    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6efc3fe2-975d-4036-b764-dfa0eed1c83c-dbus\") pod \"global-pull-secret-syncer-fbhfk\" (UID: \"6efc3fe2-975d-4036-b764-dfa0eed1c83c\") " pod="kube-system/global-pull-secret-syncer-fbhfk"
May 29 02:32:21.846358 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:21.846241    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6efc3fe2-975d-4036-b764-dfa0eed1c83c-original-pull-secret\") pod \"global-pull-secret-syncer-fbhfk\" (UID: \"6efc3fe2-975d-4036-b764-dfa0eed1c83c\") " pod="kube-system/global-pull-secret-syncer-fbhfk"
May 29 02:32:21.846358 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:21.846253    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6efc3fe2-975d-4036-b764-dfa0eed1c83c-kubelet-config\") pod \"global-pull-secret-syncer-fbhfk\" (UID: \"6efc3fe2-975d-4036-b764-dfa0eed1c83c\") " pod="kube-system/global-pull-secret-syncer-fbhfk"
May 29 02:32:21.846488 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:21.846410    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6efc3fe2-975d-4036-b764-dfa0eed1c83c-dbus\") pod \"global-pull-secret-syncer-fbhfk\" (UID: \"6efc3fe2-975d-4036-b764-dfa0eed1c83c\") " pod="kube-system/global-pull-secret-syncer-fbhfk"
May 29 02:32:21.849267 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:21.849236    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6efc3fe2-975d-4036-b764-dfa0eed1c83c-original-pull-secret\") pod \"global-pull-secret-syncer-fbhfk\" (UID: \"6efc3fe2-975d-4036-b764-dfa0eed1c83c\") " pod="kube-system/global-pull-secret-syncer-fbhfk"
May 29 02:32:21.965546 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:21.965450    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fbhfk"
May 29 02:32:22.091272 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:22.091246    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fbhfk"]
May 29 02:32:22.093243 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:32:22.093216    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6efc3fe2_975d_4036_b764_dfa0eed1c83c.slice/crio-5c0237357736d641857bdb6ccec1b297a810fa4df124307b78c9df8fbfacc46b WatchSource:0}: Error finding container 5c0237357736d641857bdb6ccec1b297a810fa4df124307b78c9df8fbfacc46b: Status 404 returned error can't find the container with id 5c0237357736d641857bdb6ccec1b297a810fa4df124307b78c9df8fbfacc46b
May 29 02:32:22.603510 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:22.603475    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fbhfk" event={"ID":"6efc3fe2-975d-4036-b764-dfa0eed1c83c","Type":"ContainerStarted","Data":"5c0237357736d641857bdb6ccec1b297a810fa4df124307b78c9df8fbfacc46b"}
May 29 02:32:26.615403 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:26.615365    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fbhfk" event={"ID":"6efc3fe2-975d-4036-b764-dfa0eed1c83c","Type":"ContainerStarted","Data":"e91dadf4b4e24ce1ede6b5f65a0a7ecaba990328158a63121c06ceca308e6151"}
May 29 02:32:39.601759 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:39.601736    2553 kubelet.go:1628] "Image garbage collection succeeded"
May 29 02:32:51.193446 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:51.193402    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:32:51.209362 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:51.209339    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:32:51.238686 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:51.238624    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fbhfk" podStartSLOduration=26.424887635 podStartE2EDuration="30.238593021s" podCreationTimestamp="2026-05-29 02:32:21 +0000 UTC" firstStartedPulling="2026-05-29 02:32:22.094896914 +0000 UTC m=+282.958509424" lastFinishedPulling="2026-05-29 02:32:25.908602298 +0000 UTC m=+286.772214810" observedRunningTime="2026-05-29 02:32:26.634259783 +0000 UTC m=+287.497872314" watchObservedRunningTime="2026-05-29 02:32:51.238593021 +0000 UTC m=+312.102205547"
May 29 02:32:51.704171 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:32:51.704140    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 29 02:33:51.370038 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.369939    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"]
May 29 02:33:51.372730 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.372712    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.375210 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.375185    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
May 29 02:33:51.375370 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.375191    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kf466\""
May 29 02:33:51.375370 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.375191    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
May 29 02:33:51.375519 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.375372    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
May 29 02:33:51.375519 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.375268    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
May 29 02:33:51.392318 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.392288    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"]
May 29 02:33:51.447593 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.447547    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2974d6c0-41cb-4276-9890-a3b55af1f299-webhook-cert\") pod \"opendatahub-operator-controller-manager-784f8c8bfd-ggzqr\" (UID: \"2974d6c0-41cb-4276-9890-a3b55af1f299\") " pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.447825 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.447605    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn5sl\" (UniqueName: \"kubernetes.io/projected/2974d6c0-41cb-4276-9890-a3b55af1f299-kube-api-access-rn5sl\") pod \"opendatahub-operator-controller-manager-784f8c8bfd-ggzqr\" (UID: \"2974d6c0-41cb-4276-9890-a3b55af1f299\") " pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.447825 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.447633    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2974d6c0-41cb-4276-9890-a3b55af1f299-tmp\") pod \"opendatahub-operator-controller-manager-784f8c8bfd-ggzqr\" (UID: \"2974d6c0-41cb-4276-9890-a3b55af1f299\") " pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.447825 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.447679    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2974d6c0-41cb-4276-9890-a3b55af1f299-apiservice-cert\") pod \"opendatahub-operator-controller-manager-784f8c8bfd-ggzqr\" (UID: \"2974d6c0-41cb-4276-9890-a3b55af1f299\") " pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.447825 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.447736    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2974d6c0-41cb-4276-9890-a3b55af1f299-manifests\") pod \"opendatahub-operator-controller-manager-784f8c8bfd-ggzqr\" (UID: \"2974d6c0-41cb-4276-9890-a3b55af1f299\") " pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.548706 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.548665    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2974d6c0-41cb-4276-9890-a3b55af1f299-apiservice-cert\") pod \"opendatahub-operator-controller-manager-784f8c8bfd-ggzqr\" (UID: \"2974d6c0-41cb-4276-9890-a3b55af1f299\") " pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.548857 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.548719    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2974d6c0-41cb-4276-9890-a3b55af1f299-manifests\") pod \"opendatahub-operator-controller-manager-784f8c8bfd-ggzqr\" (UID: \"2974d6c0-41cb-4276-9890-a3b55af1f299\") " pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.548857 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.548773    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2974d6c0-41cb-4276-9890-a3b55af1f299-webhook-cert\") pod \"opendatahub-operator-controller-manager-784f8c8bfd-ggzqr\" (UID: \"2974d6c0-41cb-4276-9890-a3b55af1f299\") " pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.548857 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.548828    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rn5sl\" (UniqueName: \"kubernetes.io/projected/2974d6c0-41cb-4276-9890-a3b55af1f299-kube-api-access-rn5sl\") pod \"opendatahub-operator-controller-manager-784f8c8bfd-ggzqr\" (UID: \"2974d6c0-41cb-4276-9890-a3b55af1f299\") " pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.548962 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.548871    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2974d6c0-41cb-4276-9890-a3b55af1f299-tmp\") pod \"opendatahub-operator-controller-manager-784f8c8bfd-ggzqr\" (UID: \"2974d6c0-41cb-4276-9890-a3b55af1f299\") " pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.549203 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.549176    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2974d6c0-41cb-4276-9890-a3b55af1f299-manifests\") pod \"opendatahub-operator-controller-manager-784f8c8bfd-ggzqr\" (UID: \"2974d6c0-41cb-4276-9890-a3b55af1f299\") " pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.551474 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.551445    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2974d6c0-41cb-4276-9890-a3b55af1f299-tmp\") pod \"opendatahub-operator-controller-manager-784f8c8bfd-ggzqr\" (UID: \"2974d6c0-41cb-4276-9890-a3b55af1f299\") " pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.551615 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.551596    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2974d6c0-41cb-4276-9890-a3b55af1f299-apiservice-cert\") pod \"opendatahub-operator-controller-manager-784f8c8bfd-ggzqr\" (UID: \"2974d6c0-41cb-4276-9890-a3b55af1f299\") " pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.551699 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.551614    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2974d6c0-41cb-4276-9890-a3b55af1f299-webhook-cert\") pod \"opendatahub-operator-controller-manager-784f8c8bfd-ggzqr\" (UID: \"2974d6c0-41cb-4276-9890-a3b55af1f299\") " pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.558718 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.558626    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn5sl\" (UniqueName: \"kubernetes.io/projected/2974d6c0-41cb-4276-9890-a3b55af1f299-kube-api-access-rn5sl\") pod \"opendatahub-operator-controller-manager-784f8c8bfd-ggzqr\" (UID: \"2974d6c0-41cb-4276-9890-a3b55af1f299\") " pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.683158 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.683071    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:51.853470 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:33:51.852422    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2974d6c0_41cb_4276_9890_a3b55af1f299.slice/crio-b84fd3da79c1b1bde510e33c8c24c5b417c451f5dd7badeb234bb1eba1af2794 WatchSource:0}: Error finding container b84fd3da79c1b1bde510e33c8c24c5b417c451f5dd7badeb234bb1eba1af2794: Status 404 returned error can't find the container with id b84fd3da79c1b1bde510e33c8c24c5b417c451f5dd7badeb234bb1eba1af2794
May 29 02:33:51.853815 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.853786    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"]
May 29 02:33:51.854439 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:51.854418    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 02:33:52.858320 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:52.858279    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr" event={"ID":"2974d6c0-41cb-4276-9890-a3b55af1f299","Type":"ContainerStarted","Data":"b84fd3da79c1b1bde510e33c8c24c5b417c451f5dd7badeb234bb1eba1af2794"}
May 29 02:33:55.868884 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:55.868837    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr" event={"ID":"2974d6c0-41cb-4276-9890-a3b55af1f299","Type":"ContainerStarted","Data":"6f576b2106af6e989ce61b2bfd382be2ce647791d45509da451202993d9c4535"}
May 29 02:33:56.873375 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:56.873339    2553 generic.go:358] "Generic (PLEG): container finished" podID="2974d6c0-41cb-4276-9890-a3b55af1f299" containerID="6f576b2106af6e989ce61b2bfd382be2ce647791d45509da451202993d9c4535" exitCode=0
May 29 02:33:56.873811 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:56.873455    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr" event={"ID":"2974d6c0-41cb-4276-9890-a3b55af1f299","Type":"ContainerDied","Data":"6f576b2106af6e989ce61b2bfd382be2ce647791d45509da451202993d9c4535"}
May 29 02:33:57.878036 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:57.877997    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr" event={"ID":"2974d6c0-41cb-4276-9890-a3b55af1f299","Type":"ContainerStarted","Data":"73c7b1220536116ddab4cb126e7622cda490a17b03f2b3359aa013767e00ff77"}
May 29 02:33:57.878559 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:57.878148    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:33:57.902044 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:33:57.901990    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr" podStartSLOduration=1.589687263 podStartE2EDuration="6.901974707s" podCreationTimestamp="2026-05-29 02:33:51 +0000 UTC" firstStartedPulling="2026-05-29 02:33:51.854597423 +0000 UTC m=+372.718209939" lastFinishedPulling="2026-05-29 02:33:57.166884874 +0000 UTC m=+378.030497383" observedRunningTime="2026-05-29 02:33:57.900783147 +0000 UTC m=+378.764395680" watchObservedRunningTime="2026-05-29 02:33:57.901974707 +0000 UTC m=+378.765587239"
May 29 02:34:08.883160 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:08.883129    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-784f8c8bfd-ggzqr"
May 29 02:34:11.544486 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.544447    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"]
May 29 02:34:11.554256 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.554220    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"
May 29 02:34:11.555529 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.555486    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"]
May 29 02:34:11.557119 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.557099    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
May 29 02:34:11.557217 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.557098    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
May 29 02:34:11.557715 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.557701    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
May 29 02:34:11.558015 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.557998    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
May 29 02:34:11.558085 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.558043    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
May 29 02:34:11.558164 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.558144    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tgrth\""
May 29 02:34:11.723880 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.719490    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/06b36631-9a0a-4467-a1f6-48bff9a4da97-metrics-cert\") pod \"lws-controller-manager-784859b99c-vm9hx\" (UID: \"06b36631-9a0a-4467-a1f6-48bff9a4da97\") " pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"
May 29 02:34:11.723880 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.719588    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/06b36631-9a0a-4467-a1f6-48bff9a4da97-manager-config\") pod \"lws-controller-manager-784859b99c-vm9hx\" (UID: \"06b36631-9a0a-4467-a1f6-48bff9a4da97\") " pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"
May 29 02:34:11.723880 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.719677    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06b36631-9a0a-4467-a1f6-48bff9a4da97-cert\") pod \"lws-controller-manager-784859b99c-vm9hx\" (UID: \"06b36631-9a0a-4467-a1f6-48bff9a4da97\") " pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"
May 29 02:34:11.723880 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.719751    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk2dd\" (UniqueName: \"kubernetes.io/projected/06b36631-9a0a-4467-a1f6-48bff9a4da97-kube-api-access-pk2dd\") pod \"lws-controller-manager-784859b99c-vm9hx\" (UID: \"06b36631-9a0a-4467-a1f6-48bff9a4da97\") " pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"
May 29 02:34:11.820887 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.820799    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/06b36631-9a0a-4467-a1f6-48bff9a4da97-metrics-cert\") pod \"lws-controller-manager-784859b99c-vm9hx\" (UID: \"06b36631-9a0a-4467-a1f6-48bff9a4da97\") " pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"
May 29 02:34:11.820887 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.820860    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/06b36631-9a0a-4467-a1f6-48bff9a4da97-manager-config\") pod \"lws-controller-manager-784859b99c-vm9hx\" (UID: \"06b36631-9a0a-4467-a1f6-48bff9a4da97\") " pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"
May 29 02:34:11.821100 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.820891    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06b36631-9a0a-4467-a1f6-48bff9a4da97-cert\") pod \"lws-controller-manager-784859b99c-vm9hx\" (UID: \"06b36631-9a0a-4467-a1f6-48bff9a4da97\") " pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"
May 29 02:34:11.821100 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.820931    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pk2dd\" (UniqueName: \"kubernetes.io/projected/06b36631-9a0a-4467-a1f6-48bff9a4da97-kube-api-access-pk2dd\") pod \"lws-controller-manager-784859b99c-vm9hx\" (UID: \"06b36631-9a0a-4467-a1f6-48bff9a4da97\") " pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"
May 29 02:34:11.821567 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.821540    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/06b36631-9a0a-4467-a1f6-48bff9a4da97-manager-config\") pod \"lws-controller-manager-784859b99c-vm9hx\" (UID: \"06b36631-9a0a-4467-a1f6-48bff9a4da97\") " pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"
May 29 02:34:11.823784 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.823760    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06b36631-9a0a-4467-a1f6-48bff9a4da97-cert\") pod \"lws-controller-manager-784859b99c-vm9hx\" (UID: \"06b36631-9a0a-4467-a1f6-48bff9a4da97\") " pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"
May 29 02:34:11.823898 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.823812    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/06b36631-9a0a-4467-a1f6-48bff9a4da97-metrics-cert\") pod \"lws-controller-manager-784859b99c-vm9hx\" (UID: \"06b36631-9a0a-4467-a1f6-48bff9a4da97\") " pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"
May 29 02:34:11.829692 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.829667    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk2dd\" (UniqueName: \"kubernetes.io/projected/06b36631-9a0a-4467-a1f6-48bff9a4da97-kube-api-access-pk2dd\") pod \"lws-controller-manager-784859b99c-vm9hx\" (UID: \"06b36631-9a0a-4467-a1f6-48bff9a4da97\") " pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"
May 29 02:34:11.864702 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:11.864637    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"
May 29 02:34:12.000207 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:12.000184    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"]
May 29 02:34:12.002850 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:34:12.002821    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06b36631_9a0a_4467_a1f6_48bff9a4da97.slice/crio-7923ae5d4cd842b6a6dca489f83c9dcbbaeadcb98ed72d5241ac04e6a0b796f4 WatchSource:0}: Error finding container 7923ae5d4cd842b6a6dca489f83c9dcbbaeadcb98ed72d5241ac04e6a0b796f4: Status 404 returned error can't find the container with id 7923ae5d4cd842b6a6dca489f83c9dcbbaeadcb98ed72d5241ac04e6a0b796f4
May 29 02:34:12.923105 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:12.923062    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx" event={"ID":"06b36631-9a0a-4467-a1f6-48bff9a4da97","Type":"ContainerStarted","Data":"7923ae5d4cd842b6a6dca489f83c9dcbbaeadcb98ed72d5241ac04e6a0b796f4"}
May 29 02:34:14.930615 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:14.930577    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx" event={"ID":"06b36631-9a0a-4467-a1f6-48bff9a4da97","Type":"ContainerStarted","Data":"0a8efdfed75854de038163df38e7425d71bde8648c98bf6c8c11dc4cf66502da"}
May 29 02:34:14.930615 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:14.930625    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"
May 29 02:34:14.947874 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:14.947830    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx" podStartSLOduration=1.217096481 podStartE2EDuration="3.947816403s" podCreationTimestamp="2026-05-29 02:34:11 +0000 UTC" firstStartedPulling="2026-05-29 02:34:12.00461563 +0000 UTC m=+392.868228139" lastFinishedPulling="2026-05-29 02:34:14.735335547 +0000 UTC m=+395.598948061" observedRunningTime="2026-05-29 02:34:14.947187802 +0000 UTC m=+395.810800336" watchObservedRunningTime="2026-05-29 02:34:14.947816403 +0000 UTC m=+395.811428925"
May 29 02:34:25.936736 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:25.936706    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-784859b99c-vm9hx"
May 29 02:34:28.964966 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:28.964925    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-55b468f665-9jlkr"]
May 29 02:34:28.968040 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:28.968020    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-55b468f665-9jlkr"
May 29 02:34:28.986718 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:28.986696    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\""
May 29 02:34:28.992995 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:28.992978    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7nqz7\""
May 29 02:34:29.012375 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:29.012347    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-55b468f665-9jlkr"]
May 29 02:34:29.068764 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:29.068734    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/deaaf730-c1d8-44c6-811a-e72ce67ed1c1-redirect-config\") pod \"dashboard-redirect-55b468f665-9jlkr\" (UID: \"deaaf730-c1d8-44c6-811a-e72ce67ed1c1\") " pod="opendatahub/dashboard-redirect-55b468f665-9jlkr"
May 29 02:34:29.068917 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:29.068777    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmqzt\" (UniqueName: \"kubernetes.io/projected/deaaf730-c1d8-44c6-811a-e72ce67ed1c1-kube-api-access-hmqzt\") pod \"dashboard-redirect-55b468f665-9jlkr\" (UID: \"deaaf730-c1d8-44c6-811a-e72ce67ed1c1\") " pod="opendatahub/dashboard-redirect-55b468f665-9jlkr"
May 29 02:34:29.170074 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:29.170041    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hmqzt\" (UniqueName: \"kubernetes.io/projected/deaaf730-c1d8-44c6-811a-e72ce67ed1c1-kube-api-access-hmqzt\") pod \"dashboard-redirect-55b468f665-9jlkr\" (UID: \"deaaf730-c1d8-44c6-811a-e72ce67ed1c1\") " pod="opendatahub/dashboard-redirect-55b468f665-9jlkr"
May 29 02:34:29.170236 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:29.170125    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/deaaf730-c1d8-44c6-811a-e72ce67ed1c1-redirect-config\") pod \"dashboard-redirect-55b468f665-9jlkr\" (UID: \"deaaf730-c1d8-44c6-811a-e72ce67ed1c1\") " pod="opendatahub/dashboard-redirect-55b468f665-9jlkr"
May 29 02:34:29.170793 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:29.170775    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/deaaf730-c1d8-44c6-811a-e72ce67ed1c1-redirect-config\") pod \"dashboard-redirect-55b468f665-9jlkr\" (UID: \"deaaf730-c1d8-44c6-811a-e72ce67ed1c1\") " pod="opendatahub/dashboard-redirect-55b468f665-9jlkr"
May 29 02:34:29.204151 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:29.204124    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmqzt\" (UniqueName: \"kubernetes.io/projected/deaaf730-c1d8-44c6-811a-e72ce67ed1c1-kube-api-access-hmqzt\") pod \"dashboard-redirect-55b468f665-9jlkr\" (UID: \"deaaf730-c1d8-44c6-811a-e72ce67ed1c1\") " pod="opendatahub/dashboard-redirect-55b468f665-9jlkr"
May 29 02:34:29.277954 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:29.277934    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-55b468f665-9jlkr"
May 29 02:34:29.423360 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:29.423337    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-55b468f665-9jlkr"]
May 29 02:34:29.425555 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:34:29.425527    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeaaf730_c1d8_44c6_811a_e72ce67ed1c1.slice/crio-6459b2ffddab50054c0c3c03ef6323806357b8e739f34f8c0b71266be6fc6924 WatchSource:0}: Error finding container 6459b2ffddab50054c0c3c03ef6323806357b8e739f34f8c0b71266be6fc6924: Status 404 returned error can't find the container with id 6459b2ffddab50054c0c3c03ef6323806357b8e739f34f8c0b71266be6fc6924
May 29 02:34:29.974622 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:29.974580    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-55b468f665-9jlkr" event={"ID":"deaaf730-c1d8-44c6-811a-e72ce67ed1c1","Type":"ContainerStarted","Data":"6459b2ffddab50054c0c3c03ef6323806357b8e739f34f8c0b71266be6fc6924"}
May 29 02:34:33.988693 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:33.988568    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-55b468f665-9jlkr" event={"ID":"deaaf730-c1d8-44c6-811a-e72ce67ed1c1","Type":"ContainerStarted","Data":"9bf27bfc873cb07c25d6a578a079e8ed2f9c57b730ddf0aac06461fc3ba985c1"}
May 29 02:34:34.007310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:34.007258    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-55b468f665-9jlkr" podStartSLOduration=1.756207573 podStartE2EDuration="6.00724294s" podCreationTimestamp="2026-05-29 02:34:28 +0000 UTC" firstStartedPulling="2026-05-29 02:34:29.42678741 +0000 UTC m=+410.290399920" lastFinishedPulling="2026-05-29 02:34:33.677822776 +0000 UTC m=+414.541435287" observedRunningTime="2026-05-29 02:34:34.005598593 +0000 UTC m=+414.869211126" watchObservedRunningTime="2026-05-29 02:34:34.00724294 +0000 UTC m=+414.870855472"
May 29 02:34:51.058699 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.058643    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"]
May 29 02:34:51.066934 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.066910    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.069701 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.069675    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
May 29 02:34:51.069850 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.069757    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
May 29 02:34:51.069850 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.069780    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
May 29 02:34:51.069850 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.069790    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-vbfgv\""
May 29 02:34:51.073795 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.073770    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"]
May 29 02:34:51.155970 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.155934    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2120d8c1-9172-44cc-af86-c23c1ae2580e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.155970 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.155975    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkm9d\" (UniqueName: \"kubernetes.io/projected/2120d8c1-9172-44cc-af86-c23c1ae2580e-kube-api-access-fkm9d\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.156209 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.156001    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2120d8c1-9172-44cc-af86-c23c1ae2580e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.156209 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.156049    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2120d8c1-9172-44cc-af86-c23c1ae2580e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.156209 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.156080    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2120d8c1-9172-44cc-af86-c23c1ae2580e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.156209 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.156120    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2120d8c1-9172-44cc-af86-c23c1ae2580e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.156209 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.156195    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2120d8c1-9172-44cc-af86-c23c1ae2580e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.156416 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.156242    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2120d8c1-9172-44cc-af86-c23c1ae2580e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.156416 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.156280    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2120d8c1-9172-44cc-af86-c23c1ae2580e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.257253 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.257221    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2120d8c1-9172-44cc-af86-c23c1ae2580e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.257452 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.257266    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2120d8c1-9172-44cc-af86-c23c1ae2580e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.257452 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.257287    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2120d8c1-9172-44cc-af86-c23c1ae2580e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.257452 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.257307    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2120d8c1-9172-44cc-af86-c23c1ae2580e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.257452 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.257330    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fkm9d\" (UniqueName: \"kubernetes.io/projected/2120d8c1-9172-44cc-af86-c23c1ae2580e-kube-api-access-fkm9d\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.257452 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.257350    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2120d8c1-9172-44cc-af86-c23c1ae2580e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.257452 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.257368    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2120d8c1-9172-44cc-af86-c23c1ae2580e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.257783 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.257521    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2120d8c1-9172-44cc-af86-c23c1ae2580e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.257783 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.257584    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2120d8c1-9172-44cc-af86-c23c1ae2580e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.257783 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.257704    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2120d8c1-9172-44cc-af86-c23c1ae2580e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.257900 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.257783    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2120d8c1-9172-44cc-af86-c23c1ae2580e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.257900 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.257860    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2120d8c1-9172-44cc-af86-c23c1ae2580e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.257900 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.257884    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2120d8c1-9172-44cc-af86-c23c1ae2580e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.258317 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.258299    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2120d8c1-9172-44cc-af86-c23c1ae2580e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.259694 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.259672    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2120d8c1-9172-44cc-af86-c23c1ae2580e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.260057 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.260041    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2120d8c1-9172-44cc-af86-c23c1ae2580e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.265636 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.265607    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2120d8c1-9172-44cc-af86-c23c1ae2580e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.265881 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.265864    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkm9d\" (UniqueName: \"kubernetes.io/projected/2120d8c1-9172-44cc-af86-c23c1ae2580e-kube-api-access-fkm9d\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8\" (UID: \"2120d8c1-9172-44cc-af86-c23c1ae2580e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.379063 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.378969    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:51.515282 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:51.515250    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"]
May 29 02:34:51.517692 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:34:51.517643    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2120d8c1_9172_44cc_af86_c23c1ae2580e.slice/crio-182ee87388d3372b09d778ec9cd19e1e5371a886dd83047dafe474c5ded12d54 WatchSource:0}: Error finding container 182ee87388d3372b09d778ec9cd19e1e5371a886dd83047dafe474c5ded12d54: Status 404 returned error can't find the container with id 182ee87388d3372b09d778ec9cd19e1e5371a886dd83047dafe474c5ded12d54
May 29 02:34:52.040622 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:52.040582    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8" event={"ID":"2120d8c1-9172-44cc-af86-c23c1ae2580e","Type":"ContainerStarted","Data":"182ee87388d3372b09d778ec9cd19e1e5371a886dd83047dafe474c5ded12d54"}
May 29 02:34:54.489402 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:54.489358    2553 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
May 29 02:34:54.489694 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:54.489456    2553 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
May 29 02:34:54.489694 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:54.489487    2553 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
May 29 02:34:55.052046 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:55.052003    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8" event={"ID":"2120d8c1-9172-44cc-af86-c23c1ae2580e","Type":"ContainerStarted","Data":"4ee3f4003266b3f79959f9a41cb1ba274409710c2eb9ed7d54d35177c72dbff1"}
May 29 02:34:55.075977 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:55.075927    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8" podStartSLOduration=1.106536546 podStartE2EDuration="4.075911563s" podCreationTimestamp="2026-05-29 02:34:51 +0000 UTC" firstStartedPulling="2026-05-29 02:34:51.519752111 +0000 UTC m=+432.383364621" lastFinishedPulling="2026-05-29 02:34:54.489127125 +0000 UTC m=+435.352739638" observedRunningTime="2026-05-29 02:34:55.074261989 +0000 UTC m=+435.937874521" watchObservedRunningTime="2026-05-29 02:34:55.075911563 +0000 UTC m=+435.939524099"
May 29 02:34:55.380110 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:55.380014    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:55.384676 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:55.384625    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:56.055177 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:56.055144    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:34:56.056356 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:34:56.056335    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8"
May 29 02:35:11.994301 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:11.994261    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9mggg"]
May 29 02:35:12.019353 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.019320    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9mggg"]
May 29 02:35:12.019511 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.019440    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9mggg"
May 29 02:35:12.022162 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.022130    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
May 29 02:35:12.022307 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.022292    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-lgtwd\""
May 29 02:35:12.023230 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.023205    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
May 29 02:35:12.132057 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.132019    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbjqc\" (UniqueName: \"kubernetes.io/projected/fc2d7d99-1abe-42cb-a4de-824f00317edc-kube-api-access-vbjqc\") pod \"kuadrant-operator-catalog-9mggg\" (UID: \"fc2d7d99-1abe-42cb-a4de-824f00317edc\") " pod="kuadrant-system/kuadrant-operator-catalog-9mggg"
May 29 02:35:12.233078 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.233019    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vbjqc\" (UniqueName: \"kubernetes.io/projected/fc2d7d99-1abe-42cb-a4de-824f00317edc-kube-api-access-vbjqc\") pod \"kuadrant-operator-catalog-9mggg\" (UID: \"fc2d7d99-1abe-42cb-a4de-824f00317edc\") " pod="kuadrant-system/kuadrant-operator-catalog-9mggg"
May 29 02:35:12.241725 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.241695    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbjqc\" (UniqueName: \"kubernetes.io/projected/fc2d7d99-1abe-42cb-a4de-824f00317edc-kube-api-access-vbjqc\") pod \"kuadrant-operator-catalog-9mggg\" (UID: \"fc2d7d99-1abe-42cb-a4de-824f00317edc\") " pod="kuadrant-system/kuadrant-operator-catalog-9mggg"
May 29 02:35:12.329644 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.329601    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9mggg"
May 29 02:35:12.353142 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.353112    2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9mggg"]
May 29 02:35:12.458370 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.458343    2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9mggg"]
May 29 02:35:12.460861 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:35:12.460827    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc2d7d99_1abe_42cb_a4de_824f00317edc.slice/crio-c75a144339160276acaaf634d675701bb8a185cf9d0ca6aee86a7ac87963c58f WatchSource:0}: Error finding container c75a144339160276acaaf634d675701bb8a185cf9d0ca6aee86a7ac87963c58f: Status 404 returned error can't find the container with id c75a144339160276acaaf634d675701bb8a185cf9d0ca6aee86a7ac87963c58f
May 29 02:35:12.563414 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.563380    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2p2j2"]
May 29 02:35:12.567945 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.567927    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2p2j2"
May 29 02:35:12.574466 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.574436    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2p2j2"]
May 29 02:35:12.736444 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.736354    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmkll\" (UniqueName: \"kubernetes.io/projected/92b38dd5-c056-4fc4-a772-46073f270636-kube-api-access-dmkll\") pod \"kuadrant-operator-catalog-2p2j2\" (UID: \"92b38dd5-c056-4fc4-a772-46073f270636\") " pod="kuadrant-system/kuadrant-operator-catalog-2p2j2"
May 29 02:35:12.837365 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.837326    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dmkll\" (UniqueName: \"kubernetes.io/projected/92b38dd5-c056-4fc4-a772-46073f270636-kube-api-access-dmkll\") pod \"kuadrant-operator-catalog-2p2j2\" (UID: \"92b38dd5-c056-4fc4-a772-46073f270636\") " pod="kuadrant-system/kuadrant-operator-catalog-2p2j2"
May 29 02:35:12.846047 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.846019    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmkll\" (UniqueName: \"kubernetes.io/projected/92b38dd5-c056-4fc4-a772-46073f270636-kube-api-access-dmkll\") pod \"kuadrant-operator-catalog-2p2j2\" (UID: \"92b38dd5-c056-4fc4-a772-46073f270636\") " pod="kuadrant-system/kuadrant-operator-catalog-2p2j2"
May 29 02:35:12.878833 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:12.878799    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2p2j2"
May 29 02:35:13.009056 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:13.008952    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2p2j2"]
May 29 02:35:13.046205 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:35:13.046172    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92b38dd5_c056_4fc4_a772_46073f270636.slice/crio-b46e80b1b9bf73106164380b741e7f947cb7d477d15e0972869e4828c7661810 WatchSource:0}: Error finding container b46e80b1b9bf73106164380b741e7f947cb7d477d15e0972869e4828c7661810: Status 404 returned error can't find the container with id b46e80b1b9bf73106164380b741e7f947cb7d477d15e0972869e4828c7661810
May 29 02:35:13.116743 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:13.116687    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2p2j2" event={"ID":"92b38dd5-c056-4fc4-a772-46073f270636","Type":"ContainerStarted","Data":"b46e80b1b9bf73106164380b741e7f947cb7d477d15e0972869e4828c7661810"}
May 29 02:35:13.117851 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:13.117820    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9mggg" event={"ID":"fc2d7d99-1abe-42cb-a4de-824f00317edc","Type":"ContainerStarted","Data":"c75a144339160276acaaf634d675701bb8a185cf9d0ca6aee86a7ac87963c58f"}
May 29 02:35:15.126932 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:15.126889    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2p2j2" event={"ID":"92b38dd5-c056-4fc4-a772-46073f270636","Type":"ContainerStarted","Data":"851ae44a894feadaeb4ed2a44c00a33ce8d81a4086ed43a75b695e8c0efa260a"}
May 29 02:35:15.128274 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:15.128244    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9mggg" event={"ID":"fc2d7d99-1abe-42cb-a4de-824f00317edc","Type":"ContainerStarted","Data":"a703ebfdbf2625c2c474da0d3e440320738b930017be11f03791352661f4ac46"}
May 29 02:35:15.128397 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:15.128358    2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-9mggg" podUID="fc2d7d99-1abe-42cb-a4de-824f00317edc" containerName="registry-server" containerID="cri-o://a703ebfdbf2625c2c474da0d3e440320738b930017be11f03791352661f4ac46" gracePeriod=2
May 29 02:35:15.143818 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:15.143769    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2p2j2" podStartSLOduration=1.5208693279999999 podStartE2EDuration="3.143750682s" podCreationTimestamp="2026-05-29 02:35:12 +0000 UTC" firstStartedPulling="2026-05-29 02:35:13.047492846 +0000 UTC m=+453.911105358" lastFinishedPulling="2026-05-29 02:35:14.670374202 +0000 UTC m=+455.533986712" observedRunningTime="2026-05-29 02:35:15.14236297 +0000 UTC m=+456.005975503" watchObservedRunningTime="2026-05-29 02:35:15.143750682 +0000 UTC m=+456.007363215"
May 29 02:35:15.157569 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:15.157524    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-9mggg" podStartSLOduration=1.952341251 podStartE2EDuration="4.157511049s" podCreationTimestamp="2026-05-29 02:35:11 +0000 UTC" firstStartedPulling="2026-05-29 02:35:12.462256193 +0000 UTC m=+453.325868702" lastFinishedPulling="2026-05-29 02:35:14.667425991 +0000 UTC m=+455.531038500" observedRunningTime="2026-05-29 02:35:15.156522739 +0000 UTC m=+456.020135270" watchObservedRunningTime="2026-05-29 02:35:15.157511049 +0000 UTC m=+456.021123581"
May 29 02:35:15.373162 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:15.373140    2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9mggg"
May 29 02:35:15.461721 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:15.461630    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbjqc\" (UniqueName: \"kubernetes.io/projected/fc2d7d99-1abe-42cb-a4de-824f00317edc-kube-api-access-vbjqc\") pod \"fc2d7d99-1abe-42cb-a4de-824f00317edc\" (UID: \"fc2d7d99-1abe-42cb-a4de-824f00317edc\") "
May 29 02:35:15.463914 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:15.463882    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc2d7d99-1abe-42cb-a4de-824f00317edc-kube-api-access-vbjqc" (OuterVolumeSpecName: "kube-api-access-vbjqc") pod "fc2d7d99-1abe-42cb-a4de-824f00317edc" (UID: "fc2d7d99-1abe-42cb-a4de-824f00317edc"). InnerVolumeSpecName "kube-api-access-vbjqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 02:35:15.562592 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:15.562546    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbjqc\" (UniqueName: \"kubernetes.io/projected/fc2d7d99-1abe-42cb-a4de-824f00317edc-kube-api-access-vbjqc\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:35:16.132704 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:16.132645    2553 generic.go:358] "Generic (PLEG): container finished" podID="fc2d7d99-1abe-42cb-a4de-824f00317edc" containerID="a703ebfdbf2625c2c474da0d3e440320738b930017be11f03791352661f4ac46" exitCode=0
May 29 02:35:16.132704 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:16.132684    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9mggg" event={"ID":"fc2d7d99-1abe-42cb-a4de-824f00317edc","Type":"ContainerDied","Data":"a703ebfdbf2625c2c474da0d3e440320738b930017be11f03791352661f4ac46"}
May 29 02:35:16.133166 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:16.132717    2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9mggg"
May 29 02:35:16.133166 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:16.132738    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9mggg" event={"ID":"fc2d7d99-1abe-42cb-a4de-824f00317edc","Type":"ContainerDied","Data":"c75a144339160276acaaf634d675701bb8a185cf9d0ca6aee86a7ac87963c58f"}
May 29 02:35:16.133166 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:16.132764    2553 scope.go:117] "RemoveContainer" containerID="a703ebfdbf2625c2c474da0d3e440320738b930017be11f03791352661f4ac46"
May 29 02:35:16.141114 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:16.141087    2553 scope.go:117] "RemoveContainer" containerID="a703ebfdbf2625c2c474da0d3e440320738b930017be11f03791352661f4ac46"
May 29 02:35:16.141365 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:35:16.141346    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a703ebfdbf2625c2c474da0d3e440320738b930017be11f03791352661f4ac46\": container with ID starting with a703ebfdbf2625c2c474da0d3e440320738b930017be11f03791352661f4ac46 not found: ID does not exist" containerID="a703ebfdbf2625c2c474da0d3e440320738b930017be11f03791352661f4ac46"
May 29 02:35:16.141428 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:16.141377    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a703ebfdbf2625c2c474da0d3e440320738b930017be11f03791352661f4ac46"} err="failed to get container status \"a703ebfdbf2625c2c474da0d3e440320738b930017be11f03791352661f4ac46\": rpc error: code = NotFound desc = could not find container \"a703ebfdbf2625c2c474da0d3e440320738b930017be11f03791352661f4ac46\": container with ID starting with a703ebfdbf2625c2c474da0d3e440320738b930017be11f03791352661f4ac46 not found: ID does not exist"
May 29 02:35:16.149450 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:16.149423    2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9mggg"]
May 29 02:35:16.153541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:16.153519    2553 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9mggg"]
May 29 02:35:17.719951 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:17.719921    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc2d7d99-1abe-42cb-a4de-824f00317edc" path="/var/lib/kubelet/pods/fc2d7d99-1abe-42cb-a4de-824f00317edc/volumes"
May 29 02:35:22.879530 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:22.879491    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-2p2j2"
May 29 02:35:22.880042 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:22.879581    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-2p2j2"
May 29 02:35:22.901774 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:22.901747    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-2p2j2"
May 29 02:35:23.176677 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:23.176584    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-2p2j2"
May 29 02:35:43.311164 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:43.311131    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd"]
May 29 02:35:43.311552 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:43.311426    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc2d7d99-1abe-42cb-a4de-824f00317edc" containerName="registry-server"
May 29 02:35:43.311552 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:43.311438    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc2d7d99-1abe-42cb-a4de-824f00317edc" containerName="registry-server"
May 29 02:35:43.311552 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:43.311494    2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="fc2d7d99-1abe-42cb-a4de-824f00317edc" containerName="registry-server"
May 29 02:35:43.319345 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:43.319323    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd"
May 29 02:35:43.322073 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:43.321955    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-ccs9q\""
May 29 02:35:43.322764 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:43.322737    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd"]
May 29 02:35:43.391823 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:43.391791    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghppb\" (UniqueName: \"kubernetes.io/projected/1177ec93-ff7e-4e5b-9b2c-23acc16c7cee-kube-api-access-ghppb\") pod \"limitador-operator-controller-manager-85c4996f8c-gwdgd\" (UID: \"1177ec93-ff7e-4e5b-9b2c-23acc16c7cee\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd"
May 29 02:35:43.493124 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:43.493089    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ghppb\" (UniqueName: \"kubernetes.io/projected/1177ec93-ff7e-4e5b-9b2c-23acc16c7cee-kube-api-access-ghppb\") pod \"limitador-operator-controller-manager-85c4996f8c-gwdgd\" (UID: \"1177ec93-ff7e-4e5b-9b2c-23acc16c7cee\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd"
May 29 02:35:43.509302 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:43.509268    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghppb\" (UniqueName: \"kubernetes.io/projected/1177ec93-ff7e-4e5b-9b2c-23acc16c7cee-kube-api-access-ghppb\") pod \"limitador-operator-controller-manager-85c4996f8c-gwdgd\" (UID: \"1177ec93-ff7e-4e5b-9b2c-23acc16c7cee\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd"
May 29 02:35:43.630340 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:43.630234    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd"
May 29 02:35:43.762825 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:43.762796    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd"]
May 29 02:35:43.764687 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:35:43.764643    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1177ec93_ff7e_4e5b_9b2c_23acc16c7cee.slice/crio-4e8d664e7fdf9c6c7e66f347e9fb547bced8cc1773fc4659a1bf85bd5805ae07 WatchSource:0}: Error finding container 4e8d664e7fdf9c6c7e66f347e9fb547bced8cc1773fc4659a1bf85bd5805ae07: Status 404 returned error can't find the container with id 4e8d664e7fdf9c6c7e66f347e9fb547bced8cc1773fc4659a1bf85bd5805ae07
May 29 02:35:44.226363 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:44.226329    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd" event={"ID":"1177ec93-ff7e-4e5b-9b2c-23acc16c7cee","Type":"ContainerStarted","Data":"4e8d664e7fdf9c6c7e66f347e9fb547bced8cc1773fc4659a1bf85bd5805ae07"}
May 29 02:35:46.237425 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:46.237394    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd" event={"ID":"1177ec93-ff7e-4e5b-9b2c-23acc16c7cee","Type":"ContainerStarted","Data":"bbfb4c1da5c1b98631978283b8ed0ff068b450a88e00c27701d9c1715f733dc1"}
May 29 02:35:46.237855 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:46.237571    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd"
May 29 02:35:46.285516 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:46.285466    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd" podStartSLOduration=1.625134766 podStartE2EDuration="3.285451568s" podCreationTimestamp="2026-05-29 02:35:43 +0000 UTC" firstStartedPulling="2026-05-29 02:35:43.76670667 +0000 UTC m=+484.630319181" lastFinishedPulling="2026-05-29 02:35:45.427023473 +0000 UTC m=+486.290635983" observedRunningTime="2026-05-29 02:35:46.28321348 +0000 UTC m=+487.146826013" watchObservedRunningTime="2026-05-29 02:35:46.285451568 +0000 UTC m=+487.149064100"
May 29 02:35:51.721839 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:51.721794    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-spg8s"]
May 29 02:35:51.729150 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:51.729126    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-spg8s"
May 29 02:35:51.731902 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:51.731864    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-c792q\""
May 29 02:35:51.732612 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:51.732584    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-spg8s"]
May 29 02:35:51.766767 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:51.766734    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp4p7\" (UniqueName: \"kubernetes.io/projected/d1135c53-67c0-448d-8413-d79be85b5c8a-kube-api-access-kp4p7\") pod \"authorino-operator-657f44b778-spg8s\" (UID: \"d1135c53-67c0-448d-8413-d79be85b5c8a\") " pod="kuadrant-system/authorino-operator-657f44b778-spg8s"
May 29 02:35:51.867241 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:51.867207    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kp4p7\" (UniqueName: \"kubernetes.io/projected/d1135c53-67c0-448d-8413-d79be85b5c8a-kube-api-access-kp4p7\") pod \"authorino-operator-657f44b778-spg8s\" (UID: \"d1135c53-67c0-448d-8413-d79be85b5c8a\") " pod="kuadrant-system/authorino-operator-657f44b778-spg8s"
May 29 02:35:51.876367 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:51.876339    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp4p7\" (UniqueName: \"kubernetes.io/projected/d1135c53-67c0-448d-8413-d79be85b5c8a-kube-api-access-kp4p7\") pod \"authorino-operator-657f44b778-spg8s\" (UID: \"d1135c53-67c0-448d-8413-d79be85b5c8a\") " pod="kuadrant-system/authorino-operator-657f44b778-spg8s"
May 29 02:35:52.040417 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:52.040376    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-spg8s"
May 29 02:35:52.168543 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:52.168497    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-spg8s"]
May 29 02:35:52.170902 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:35:52.170874    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1135c53_67c0_448d_8413_d79be85b5c8a.slice/crio-16e73e54272e5f4920fd3af95d1f6b046f5d22e66fcc10b4c60b7ee1e0ce0051 WatchSource:0}: Error finding container 16e73e54272e5f4920fd3af95d1f6b046f5d22e66fcc10b4c60b7ee1e0ce0051: Status 404 returned error can't find the container with id 16e73e54272e5f4920fd3af95d1f6b046f5d22e66fcc10b4c60b7ee1e0ce0051
May 29 02:35:52.257097 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:52.257056    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-spg8s" event={"ID":"d1135c53-67c0-448d-8413-d79be85b5c8a","Type":"ContainerStarted","Data":"16e73e54272e5f4920fd3af95d1f6b046f5d22e66fcc10b4c60b7ee1e0ce0051"}
May 29 02:35:55.269231 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:55.269194    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-spg8s" event={"ID":"d1135c53-67c0-448d-8413-d79be85b5c8a","Type":"ContainerStarted","Data":"c23ece42235af300c6c6a983d892088f47bd122c514da26628248c3ae63a53b6"}
May 29 02:35:55.269740 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:55.269356    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-spg8s"
May 29 02:35:55.289931 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:55.289885    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-spg8s" podStartSLOduration=2.118682894 podStartE2EDuration="4.289870624s" podCreationTimestamp="2026-05-29 02:35:51 +0000 UTC" firstStartedPulling="2026-05-29 02:35:52.172895112 +0000 UTC m=+493.036507622" lastFinishedPulling="2026-05-29 02:35:54.344082842 +0000 UTC m=+495.207695352" observedRunningTime="2026-05-29 02:35:55.288727613 +0000 UTC m=+496.152340144" watchObservedRunningTime="2026-05-29 02:35:55.289870624 +0000 UTC m=+496.153483397"
May 29 02:35:57.243272 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:35:57.243229    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd"
May 29 02:36:06.274925 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:06.274884    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-spg8s"
May 29 02:36:07.323175 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.323139    2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd"]
May 29 02:36:07.323679 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.323380    2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd" podUID="1177ec93-ff7e-4e5b-9b2c-23acc16c7cee" containerName="manager" containerID="cri-o://bbfb4c1da5c1b98631978283b8ed0ff068b450a88e00c27701d9c1715f733dc1" gracePeriod=2
May 29 02:36:07.330358 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.330329    2553 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd"]
May 29 02:36:07.346162 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.346136    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-5n9ff"]
May 29 02:36:07.346682 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.346622    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1177ec93-ff7e-4e5b-9b2c-23acc16c7cee" containerName="manager"
May 29 02:36:07.346682 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.346678    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1177ec93-ff7e-4e5b-9b2c-23acc16c7cee" containerName="manager"
May 29 02:36:07.346850 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.346768    2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="1177ec93-ff7e-4e5b-9b2c-23acc16c7cee" containerName="manager"
May 29 02:36:07.350089 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.350064    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-5n9ff"
May 29 02:36:07.352340 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.352314    2553 status_manager.go:895] "Failed to get status for pod" podUID="1177ec93-ff7e-4e5b-9b2c-23acc16c7cee" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd" err="pods \"limitador-operator-controller-manager-85c4996f8c-gwdgd\" is forbidden: User \"system:node:ip-10-0-133-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-245.ec2.internal' and this object"
May 29 02:36:07.359040 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.359016    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-5n9ff"]
May 29 02:36:07.500241 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.500211    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85n9m\" (UniqueName: \"kubernetes.io/projected/71924069-77b1-4a35-88cf-5e5207d98493-kube-api-access-85n9m\") pod \"limitador-operator-controller-manager-85c4996f8c-5n9ff\" (UID: \"71924069-77b1-4a35-88cf-5e5207d98493\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-5n9ff"
May 29 02:36:07.566185 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.566158    2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd"
May 29 02:36:07.568701 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.568648    2553 status_manager.go:895] "Failed to get status for pod" podUID="1177ec93-ff7e-4e5b-9b2c-23acc16c7cee" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd" err="pods \"limitador-operator-controller-manager-85c4996f8c-gwdgd\" is forbidden: User \"system:node:ip-10-0-133-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-245.ec2.internal' and this object"
May 29 02:36:07.601210 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.601129    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-85n9m\" (UniqueName: \"kubernetes.io/projected/71924069-77b1-4a35-88cf-5e5207d98493-kube-api-access-85n9m\") pod \"limitador-operator-controller-manager-85c4996f8c-5n9ff\" (UID: \"71924069-77b1-4a35-88cf-5e5207d98493\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-5n9ff"
May 29 02:36:07.612317 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.612287    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85n9m\" (UniqueName: \"kubernetes.io/projected/71924069-77b1-4a35-88cf-5e5207d98493-kube-api-access-85n9m\") pod \"limitador-operator-controller-manager-85c4996f8c-5n9ff\" (UID: \"71924069-77b1-4a35-88cf-5e5207d98493\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-5n9ff"
May 29 02:36:07.697294 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.697255    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-5n9ff"
May 29 02:36:07.702258 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.702231    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghppb\" (UniqueName: \"kubernetes.io/projected/1177ec93-ff7e-4e5b-9b2c-23acc16c7cee-kube-api-access-ghppb\") pod \"1177ec93-ff7e-4e5b-9b2c-23acc16c7cee\" (UID: \"1177ec93-ff7e-4e5b-9b2c-23acc16c7cee\") "
May 29 02:36:07.704388 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.704358    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1177ec93-ff7e-4e5b-9b2c-23acc16c7cee-kube-api-access-ghppb" (OuterVolumeSpecName: "kube-api-access-ghppb") pod "1177ec93-ff7e-4e5b-9b2c-23acc16c7cee" (UID: "1177ec93-ff7e-4e5b-9b2c-23acc16c7cee"). InnerVolumeSpecName "kube-api-access-ghppb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 02:36:07.721910 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.721881    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1177ec93-ff7e-4e5b-9b2c-23acc16c7cee" path="/var/lib/kubelet/pods/1177ec93-ff7e-4e5b-9b2c-23acc16c7cee/volumes"
May 29 02:36:07.803337 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.803301    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghppb\" (UniqueName: \"kubernetes.io/projected/1177ec93-ff7e-4e5b-9b2c-23acc16c7cee-kube-api-access-ghppb\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:36:07.830864 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:07.830698    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-5n9ff"]
May 29 02:36:07.833677 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:36:07.833637    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71924069_77b1_4a35_88cf_5e5207d98493.slice/crio-4fdc7aa683b040c333976299efa95a1bd034b279057e9a0ffc22c5949f0ad86a WatchSource:0}: Error finding container 4fdc7aa683b040c333976299efa95a1bd034b279057e9a0ffc22c5949f0ad86a: Status 404 returned error can't find the container with id 4fdc7aa683b040c333976299efa95a1bd034b279057e9a0ffc22c5949f0ad86a
May 29 02:36:08.312445 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:08.312412    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-5n9ff" event={"ID":"71924069-77b1-4a35-88cf-5e5207d98493","Type":"ContainerStarted","Data":"1ee18795ba300e2e77c077bebd3ce5db223d7b68f245cde6388e25f46c51bb26"}
May 29 02:36:08.312445 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:08.312450    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-5n9ff" event={"ID":"71924069-77b1-4a35-88cf-5e5207d98493","Type":"ContainerStarted","Data":"4fdc7aa683b040c333976299efa95a1bd034b279057e9a0ffc22c5949f0ad86a"}
May 29 02:36:08.312695 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:08.312497    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-5n9ff"
May 29 02:36:08.313561 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:08.313538    2553 generic.go:358] "Generic (PLEG): container finished" podID="1177ec93-ff7e-4e5b-9b2c-23acc16c7cee" containerID="bbfb4c1da5c1b98631978283b8ed0ff068b450a88e00c27701d9c1715f733dc1" exitCode=0
May 29 02:36:08.313680 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:08.313578    2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-gwdgd"
May 29 02:36:08.313680 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:08.313617    2553 scope.go:117] "RemoveContainer" containerID="bbfb4c1da5c1b98631978283b8ed0ff068b450a88e00c27701d9c1715f733dc1"
May 29 02:36:08.323456 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:08.323438    2553 scope.go:117] "RemoveContainer" containerID="bbfb4c1da5c1b98631978283b8ed0ff068b450a88e00c27701d9c1715f733dc1"
May 29 02:36:08.323976 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:36:08.323949    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbfb4c1da5c1b98631978283b8ed0ff068b450a88e00c27701d9c1715f733dc1\": container with ID starting with bbfb4c1da5c1b98631978283b8ed0ff068b450a88e00c27701d9c1715f733dc1 not found: ID does not exist" containerID="bbfb4c1da5c1b98631978283b8ed0ff068b450a88e00c27701d9c1715f733dc1"
May 29 02:36:08.324036 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:08.323986    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbfb4c1da5c1b98631978283b8ed0ff068b450a88e00c27701d9c1715f733dc1"} err="failed to get container status \"bbfb4c1da5c1b98631978283b8ed0ff068b450a88e00c27701d9c1715f733dc1\": rpc error: code = NotFound desc = could not find container \"bbfb4c1da5c1b98631978283b8ed0ff068b450a88e00c27701d9c1715f733dc1\": container with ID starting with bbfb4c1da5c1b98631978283b8ed0ff068b450a88e00c27701d9c1715f733dc1 not found: ID does not exist"
May 29 02:36:08.337402 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:08.337357    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-5n9ff" podStartSLOduration=1.337345124 podStartE2EDuration="1.337345124s" podCreationTimestamp="2026-05-29 02:36:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 02:36:08.337332971 +0000 UTC m=+509.200945504" watchObservedRunningTime="2026-05-29 02:36:08.337345124 +0000 UTC m=+509.200957657"
May 29 02:36:19.320935 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:19.320904    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-5n9ff"
May 29 02:36:37.032318 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.032244    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"]
May 29 02:36:37.036613 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.036594    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.039083 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.039060    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-d8pp7\""
May 29 02:36:37.048861 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.048837    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"]
May 29 02:36:37.138084 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.138046    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3efd16c1-2a11-4eb6-a42f-e7326eea722a-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.138084 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.138083    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3efd16c1-2a11-4eb6-a42f-e7326eea722a-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.138291 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.138106    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3efd16c1-2a11-4eb6-a42f-e7326eea722a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.138291 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.138124    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3efd16c1-2a11-4eb6-a42f-e7326eea722a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.138291 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.138144    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2d44\" (UniqueName: \"kubernetes.io/projected/3efd16c1-2a11-4eb6-a42f-e7326eea722a-kube-api-access-j2d44\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.138291 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.138181    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3efd16c1-2a11-4eb6-a42f-e7326eea722a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.138291 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.138209    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3efd16c1-2a11-4eb6-a42f-e7326eea722a-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.138291 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.138275    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3efd16c1-2a11-4eb6-a42f-e7326eea722a-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.138471 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.138341    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3efd16c1-2a11-4eb6-a42f-e7326eea722a-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.239805 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.239766    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3efd16c1-2a11-4eb6-a42f-e7326eea722a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.239972 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.239822    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3efd16c1-2a11-4eb6-a42f-e7326eea722a-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.239972 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.239855    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3efd16c1-2a11-4eb6-a42f-e7326eea722a-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.239972 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.239900    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3efd16c1-2a11-4eb6-a42f-e7326eea722a-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.239972 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.239936    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3efd16c1-2a11-4eb6-a42f-e7326eea722a-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.239972 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.239958    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3efd16c1-2a11-4eb6-a42f-e7326eea722a-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.240211 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.239985    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3efd16c1-2a11-4eb6-a42f-e7326eea722a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.240211 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.240131    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3efd16c1-2a11-4eb6-a42f-e7326eea722a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.240211 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.240183    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j2d44\" (UniqueName: \"kubernetes.io/projected/3efd16c1-2a11-4eb6-a42f-e7326eea722a-kube-api-access-j2d44\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.240375 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.240272    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3efd16c1-2a11-4eb6-a42f-e7326eea722a-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.240732 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.240706    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3efd16c1-2a11-4eb6-a42f-e7326eea722a-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.240932 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.240759    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3efd16c1-2a11-4eb6-a42f-e7326eea722a-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.240932 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.240797    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3efd16c1-2a11-4eb6-a42f-e7326eea722a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.240932 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.240847    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3efd16c1-2a11-4eb6-a42f-e7326eea722a-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.242245 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.242228    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3efd16c1-2a11-4eb6-a42f-e7326eea722a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.242903 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.242885    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3efd16c1-2a11-4eb6-a42f-e7326eea722a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.247677 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.247632    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3efd16c1-2a11-4eb6-a42f-e7326eea722a-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.247801 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.247785    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2d44\" (UniqueName: \"kubernetes.io/projected/3efd16c1-2a11-4eb6-a42f-e7326eea722a-kube-api-access-j2d44\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-s8rdr\" (UID: \"3efd16c1-2a11-4eb6-a42f-e7326eea722a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.346720 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.346603    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:37.484864 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.484830    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"]
May 29 02:36:37.488084 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:36:37.488055    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3efd16c1_2a11_4eb6_a42f_e7326eea722a.slice/crio-1cc1ab7dca9b340cbc0b32078074b1167c99052844d465310ec9d574fe4c55b5 WatchSource:0}: Error finding container 1cc1ab7dca9b340cbc0b32078074b1167c99052844d465310ec9d574fe4c55b5: Status 404 returned error can't find the container with id 1cc1ab7dca9b340cbc0b32078074b1167c99052844d465310ec9d574fe4c55b5
May 29 02:36:37.490756 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.490727    2553 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
May 29 02:36:37.490827 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.490790    2553 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
May 29 02:36:37.490827 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:37.490818    2553 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
May 29 02:36:38.419939 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:38.419898    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr" event={"ID":"3efd16c1-2a11-4eb6-a42f-e7326eea722a","Type":"ContainerStarted","Data":"35de2f7640dc7aae8d24088bf8069a1125e6d5c78c036e0a9d728d30da6f206a"}
May 29 02:36:38.419939 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:38.419939    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr" event={"ID":"3efd16c1-2a11-4eb6-a42f-e7326eea722a","Type":"ContainerStarted","Data":"1cc1ab7dca9b340cbc0b32078074b1167c99052844d465310ec9d574fe4c55b5"}
May 29 02:36:38.440528 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:38.440470    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr" podStartSLOduration=1.440451835 podStartE2EDuration="1.440451835s" podCreationTimestamp="2026-05-29 02:36:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 02:36:38.439027118 +0000 UTC m=+539.302639651" watchObservedRunningTime="2026-05-29 02:36:38.440451835 +0000 UTC m=+539.304064371"
May 29 02:36:39.347742 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:39.347706    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:39.352507 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:39.352480    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:39.423384 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:39.423352    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:39.424309 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:39.424293    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-s8rdr"
May 29 02:36:42.298531 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:42.298500    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-mpskm"]
May 29 02:36:42.301479 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:42.301463    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-mpskm"
May 29 02:36:42.304409 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:42.304377    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-l9kwg\""
May 29 02:36:42.309298 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:42.309271    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-mpskm"]
May 29 02:36:42.388839 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:42.388810    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxvjp\" (UniqueName: \"kubernetes.io/projected/676aa51c-0d48-4737-8db4-7758bff7de59-kube-api-access-gxvjp\") pod \"authorino-f99f4b5cd-mpskm\" (UID: \"676aa51c-0d48-4737-8db4-7758bff7de59\") " pod="kuadrant-system/authorino-f99f4b5cd-mpskm"
May 29 02:36:42.490104 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:42.490071    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gxvjp\" (UniqueName: \"kubernetes.io/projected/676aa51c-0d48-4737-8db4-7758bff7de59-kube-api-access-gxvjp\") pod \"authorino-f99f4b5cd-mpskm\" (UID: \"676aa51c-0d48-4737-8db4-7758bff7de59\") " pod="kuadrant-system/authorino-f99f4b5cd-mpskm"
May 29 02:36:42.501501 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:42.501472    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxvjp\" (UniqueName: \"kubernetes.io/projected/676aa51c-0d48-4737-8db4-7758bff7de59-kube-api-access-gxvjp\") pod \"authorino-f99f4b5cd-mpskm\" (UID: \"676aa51c-0d48-4737-8db4-7758bff7de59\") " pod="kuadrant-system/authorino-f99f4b5cd-mpskm"
May 29 02:36:42.611635 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:42.611536    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-mpskm"
May 29 02:36:42.759667 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:42.759580    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-mpskm"]
May 29 02:36:42.761752 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:36:42.761724    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod676aa51c_0d48_4737_8db4_7758bff7de59.slice/crio-16f60704a27f8cb50934a57083c7d275028e32cd5da35b822e6dd8edd8a18d07 WatchSource:0}: Error finding container 16f60704a27f8cb50934a57083c7d275028e32cd5da35b822e6dd8edd8a18d07: Status 404 returned error can't find the container with id 16f60704a27f8cb50934a57083c7d275028e32cd5da35b822e6dd8edd8a18d07
May 29 02:36:43.438938 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:43.438881    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-mpskm" event={"ID":"676aa51c-0d48-4737-8db4-7758bff7de59","Type":"ContainerStarted","Data":"16f60704a27f8cb50934a57083c7d275028e32cd5da35b822e6dd8edd8a18d07"}
May 29 02:36:46.452792 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:46.452690    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-mpskm" event={"ID":"676aa51c-0d48-4737-8db4-7758bff7de59","Type":"ContainerStarted","Data":"1b3b75a1532fbe6310f86cdc8c6a4ee306bce76f9bf2bc26213745e78171ce8f"}
May 29 02:36:46.468828 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:46.468781    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-mpskm" podStartSLOduration=1.183455894 podStartE2EDuration="4.468767717s" podCreationTimestamp="2026-05-29 02:36:42 +0000 UTC" firstStartedPulling="2026-05-29 02:36:42.763005947 +0000 UTC m=+543.626618458" lastFinishedPulling="2026-05-29 02:36:46.048317767 +0000 UTC m=+546.911930281" observedRunningTime="2026-05-29 02:36:46.467840685 +0000 UTC m=+547.331453216" watchObservedRunningTime="2026-05-29 02:36:46.468767717 +0000 UTC m=+547.332380249"
May 29 02:36:48.837912 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:48.837876    2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-mpskm"]
May 29 02:36:48.838403 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:48.838109    2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-mpskm" podUID="676aa51c-0d48-4737-8db4-7758bff7de59" containerName="authorino" containerID="cri-o://1b3b75a1532fbe6310f86cdc8c6a4ee306bce76f9bf2bc26213745e78171ce8f" gracePeriod=30
May 29 02:36:49.089310 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:49.089240    2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-mpskm"
May 29 02:36:49.150706 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:49.150664    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxvjp\" (UniqueName: \"kubernetes.io/projected/676aa51c-0d48-4737-8db4-7758bff7de59-kube-api-access-gxvjp\") pod \"676aa51c-0d48-4737-8db4-7758bff7de59\" (UID: \"676aa51c-0d48-4737-8db4-7758bff7de59\") "
May 29 02:36:49.153100 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:49.153068    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/676aa51c-0d48-4737-8db4-7758bff7de59-kube-api-access-gxvjp" (OuterVolumeSpecName: "kube-api-access-gxvjp") pod "676aa51c-0d48-4737-8db4-7758bff7de59" (UID: "676aa51c-0d48-4737-8db4-7758bff7de59"). InnerVolumeSpecName "kube-api-access-gxvjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 02:36:49.251740 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:49.251689    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxvjp\" (UniqueName: \"kubernetes.io/projected/676aa51c-0d48-4737-8db4-7758bff7de59-kube-api-access-gxvjp\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:36:49.464616 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:49.464528    2553 generic.go:358] "Generic (PLEG): container finished" podID="676aa51c-0d48-4737-8db4-7758bff7de59" containerID="1b3b75a1532fbe6310f86cdc8c6a4ee306bce76f9bf2bc26213745e78171ce8f" exitCode=0
May 29 02:36:49.464616 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:49.464587    2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-mpskm"
May 29 02:36:49.465002 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:49.464613    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-mpskm" event={"ID":"676aa51c-0d48-4737-8db4-7758bff7de59","Type":"ContainerDied","Data":"1b3b75a1532fbe6310f86cdc8c6a4ee306bce76f9bf2bc26213745e78171ce8f"}
May 29 02:36:49.465002 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:49.464647    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-mpskm" event={"ID":"676aa51c-0d48-4737-8db4-7758bff7de59","Type":"ContainerDied","Data":"16f60704a27f8cb50934a57083c7d275028e32cd5da35b822e6dd8edd8a18d07"}
May 29 02:36:49.465002 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:49.464688    2553 scope.go:117] "RemoveContainer" containerID="1b3b75a1532fbe6310f86cdc8c6a4ee306bce76f9bf2bc26213745e78171ce8f"
May 29 02:36:49.473158 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:49.473141    2553 scope.go:117] "RemoveContainer" containerID="1b3b75a1532fbe6310f86cdc8c6a4ee306bce76f9bf2bc26213745e78171ce8f"
May 29 02:36:49.473424 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:36:49.473407    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b3b75a1532fbe6310f86cdc8c6a4ee306bce76f9bf2bc26213745e78171ce8f\": container with ID starting with 1b3b75a1532fbe6310f86cdc8c6a4ee306bce76f9bf2bc26213745e78171ce8f not found: ID does not exist" containerID="1b3b75a1532fbe6310f86cdc8c6a4ee306bce76f9bf2bc26213745e78171ce8f"
May 29 02:36:49.473485 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:49.473431    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b3b75a1532fbe6310f86cdc8c6a4ee306bce76f9bf2bc26213745e78171ce8f"} err="failed to get container status \"1b3b75a1532fbe6310f86cdc8c6a4ee306bce76f9bf2bc26213745e78171ce8f\": rpc error: code = NotFound desc = could not find container \"1b3b75a1532fbe6310f86cdc8c6a4ee306bce76f9bf2bc26213745e78171ce8f\": container with ID starting with 1b3b75a1532fbe6310f86cdc8c6a4ee306bce76f9bf2bc26213745e78171ce8f not found: ID does not exist"
May 29 02:36:49.487261 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:49.487233    2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-mpskm"]
May 29 02:36:49.491419 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:49.491397    2553 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-mpskm"]
May 29 02:36:49.720889 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:36:49.720812    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="676aa51c-0d48-4737-8db4-7758bff7de59" path="/var/lib/kubelet/pods/676aa51c-0d48-4737-8db4-7758bff7de59/volumes"
May 29 02:37:12.539255 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.539215    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-z2rjz"]
May 29 02:37:12.539623 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.539551    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="676aa51c-0d48-4737-8db4-7758bff7de59" containerName="authorino"
May 29 02:37:12.539623 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.539562    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="676aa51c-0d48-4737-8db4-7758bff7de59" containerName="authorino"
May 29 02:37:12.539623 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.539621    2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="676aa51c-0d48-4737-8db4-7758bff7de59" containerName="authorino"
May 29 02:37:12.542891 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.542859    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-z2rjz"
May 29 02:37:12.545294 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.545270    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-l9kwg\""
May 29 02:37:12.547153 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.547130    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-z2rjz"]
May 29 02:37:12.660166 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.660128    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z92rp\" (UniqueName: \"kubernetes.io/projected/a7a0d744-912d-444f-9646-042961850bcb-kube-api-access-z92rp\") pod \"authorino-8b475cf9f-z2rjz\" (UID: \"a7a0d744-912d-444f-9646-042961850bcb\") " pod="kuadrant-system/authorino-8b475cf9f-z2rjz"
May 29 02:37:12.760762 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.760725    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z92rp\" (UniqueName: \"kubernetes.io/projected/a7a0d744-912d-444f-9646-042961850bcb-kube-api-access-z92rp\") pod \"authorino-8b475cf9f-z2rjz\" (UID: \"a7a0d744-912d-444f-9646-042961850bcb\") " pod="kuadrant-system/authorino-8b475cf9f-z2rjz"
May 29 02:37:12.769557 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.769525    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z92rp\" (UniqueName: \"kubernetes.io/projected/a7a0d744-912d-444f-9646-042961850bcb-kube-api-access-z92rp\") pod \"authorino-8b475cf9f-z2rjz\" (UID: \"a7a0d744-912d-444f-9646-042961850bcb\") " pod="kuadrant-system/authorino-8b475cf9f-z2rjz"
May 29 02:37:12.770754 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.770730    2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-z2rjz"]
May 29 02:37:12.771003 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.770988    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-z2rjz"
May 29 02:37:12.800822 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.800757    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bffcc75dc-7bdxp"]
May 29 02:37:12.805024 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.805008    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bffcc75dc-7bdxp"
May 29 02:37:12.813624 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.813596    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bffcc75dc-7bdxp"]
May 29 02:37:12.901135 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.901109    2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-z2rjz"]
May 29 02:37:12.903289 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:37:12.903261    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7a0d744_912d_444f_9646_042961850bcb.slice/crio-4e9437f9fbeb329b8028efb2d541e3a86d45404b99b852fd78e7bb736b54b93b WatchSource:0}: Error finding container 4e9437f9fbeb329b8028efb2d541e3a86d45404b99b852fd78e7bb736b54b93b: Status 404 returned error can't find the container with id 4e9437f9fbeb329b8028efb2d541e3a86d45404b99b852fd78e7bb736b54b93b
May 29 02:37:12.962361 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:12.962323    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcpvc\" (UniqueName: \"kubernetes.io/projected/05adb50f-e57d-49ad-82ba-5f32613e5def-kube-api-access-tcpvc\") pod \"authorino-7bffcc75dc-7bdxp\" (UID: \"05adb50f-e57d-49ad-82ba-5f32613e5def\") " pod="kuadrant-system/authorino-7bffcc75dc-7bdxp"
May 29 02:37:13.063265 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.063179    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tcpvc\" (UniqueName: \"kubernetes.io/projected/05adb50f-e57d-49ad-82ba-5f32613e5def-kube-api-access-tcpvc\") pod \"authorino-7bffcc75dc-7bdxp\" (UID: \"05adb50f-e57d-49ad-82ba-5f32613e5def\") " pod="kuadrant-system/authorino-7bffcc75dc-7bdxp"
May 29 02:37:13.069385 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.069353    2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bffcc75dc-7bdxp"]
May 29 02:37:13.069622 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:37:13.069598    2553 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-tcpvc], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7bffcc75dc-7bdxp" podUID="05adb50f-e57d-49ad-82ba-5f32613e5def"
May 29 02:37:13.072426 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.072402    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcpvc\" (UniqueName: \"kubernetes.io/projected/05adb50f-e57d-49ad-82ba-5f32613e5def-kube-api-access-tcpvc\") pod \"authorino-7bffcc75dc-7bdxp\" (UID: \"05adb50f-e57d-49ad-82ba-5f32613e5def\") " pod="kuadrant-system/authorino-7bffcc75dc-7bdxp"
May 29 02:37:13.098227 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.098197    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5db5594495-8qlpd"]
May 29 02:37:13.101629 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.101607    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5db5594495-8qlpd"
May 29 02:37:13.104127 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.104107    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
May 29 02:37:13.107724 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.107703    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5db5594495-8qlpd"]
May 29 02:37:13.264341 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.264319    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec1ef264-a21e-4066-acc0-a26747b2307a-tls-cert\") pod \"authorino-5db5594495-8qlpd\" (UID: \"ec1ef264-a21e-4066-acc0-a26747b2307a\") " pod="kuadrant-system/authorino-5db5594495-8qlpd"
May 29 02:37:13.264456 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.264350    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kbql\" (UniqueName: \"kubernetes.io/projected/ec1ef264-a21e-4066-acc0-a26747b2307a-kube-api-access-9kbql\") pod \"authorino-5db5594495-8qlpd\" (UID: \"ec1ef264-a21e-4066-acc0-a26747b2307a\") " pod="kuadrant-system/authorino-5db5594495-8qlpd"
May 29 02:37:13.365043 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.364950    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec1ef264-a21e-4066-acc0-a26747b2307a-tls-cert\") pod \"authorino-5db5594495-8qlpd\" (UID: \"ec1ef264-a21e-4066-acc0-a26747b2307a\") " pod="kuadrant-system/authorino-5db5594495-8qlpd"
May 29 02:37:13.365043 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.364998    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9kbql\" (UniqueName: \"kubernetes.io/projected/ec1ef264-a21e-4066-acc0-a26747b2307a-kube-api-access-9kbql\") pod \"authorino-5db5594495-8qlpd\" (UID: \"ec1ef264-a21e-4066-acc0-a26747b2307a\") " pod="kuadrant-system/authorino-5db5594495-8qlpd"
May 29 02:37:13.367525 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.367505    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec1ef264-a21e-4066-acc0-a26747b2307a-tls-cert\") pod \"authorino-5db5594495-8qlpd\" (UID: \"ec1ef264-a21e-4066-acc0-a26747b2307a\") " pod="kuadrant-system/authorino-5db5594495-8qlpd"
May 29 02:37:13.373919 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.373900    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kbql\" (UniqueName: \"kubernetes.io/projected/ec1ef264-a21e-4066-acc0-a26747b2307a-kube-api-access-9kbql\") pod \"authorino-5db5594495-8qlpd\" (UID: \"ec1ef264-a21e-4066-acc0-a26747b2307a\") " pod="kuadrant-system/authorino-5db5594495-8qlpd"
May 29 02:37:13.411967 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.411938    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5db5594495-8qlpd"
May 29 02:37:13.539366 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.539337    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5db5594495-8qlpd"]
May 29 02:37:13.541692 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:37:13.541666    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec1ef264_a21e_4066_acc0_a26747b2307a.slice/crio-bdb192ede64d0f004db001da7ec13a3887ba5191f083053189e6c4f4d7fb0908 WatchSource:0}: Error finding container bdb192ede64d0f004db001da7ec13a3887ba5191f083053189e6c4f4d7fb0908: Status 404 returned error can't find the container with id bdb192ede64d0f004db001da7ec13a3887ba5191f083053189e6c4f4d7fb0908
May 29 02:37:13.548138 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.548111    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5db5594495-8qlpd" event={"ID":"ec1ef264-a21e-4066-acc0-a26747b2307a","Type":"ContainerStarted","Data":"bdb192ede64d0f004db001da7ec13a3887ba5191f083053189e6c4f4d7fb0908"}
May 29 02:37:13.549343 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.549328    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bffcc75dc-7bdxp"
May 29 02:37:13.549418 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.549357    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-z2rjz" event={"ID":"a7a0d744-912d-444f-9646-042961850bcb","Type":"ContainerStarted","Data":"b9e56a8ea0f9feae80d022f5ba8347fffd250fdaa0af2a2b0bbf5b26a2c4d2f4"}
May 29 02:37:13.549418 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.549384    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-z2rjz" event={"ID":"a7a0d744-912d-444f-9646-042961850bcb","Type":"ContainerStarted","Data":"4e9437f9fbeb329b8028efb2d541e3a86d45404b99b852fd78e7bb736b54b93b"}
May 29 02:37:13.549418 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.549388    2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-8b475cf9f-z2rjz" podUID="a7a0d744-912d-444f-9646-042961850bcb" containerName="authorino" containerID="cri-o://b9e56a8ea0f9feae80d022f5ba8347fffd250fdaa0af2a2b0bbf5b26a2c4d2f4" gracePeriod=30
May 29 02:37:13.554582 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.554384    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bffcc75dc-7bdxp"
May 29 02:37:13.564469 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.564424    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-8b475cf9f-z2rjz" podStartSLOduration=1.263671915 podStartE2EDuration="1.564412278s" podCreationTimestamp="2026-05-29 02:37:12 +0000 UTC" firstStartedPulling="2026-05-29 02:37:12.90438282 +0000 UTC m=+573.767995330" lastFinishedPulling="2026-05-29 02:37:13.205123182 +0000 UTC m=+574.068735693" observedRunningTime="2026-05-29 02:37:13.562844275 +0000 UTC m=+574.426456820" watchObservedRunningTime="2026-05-29 02:37:13.564412278 +0000 UTC m=+574.428024809"
May 29 02:37:13.667419 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.667386    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcpvc\" (UniqueName: \"kubernetes.io/projected/05adb50f-e57d-49ad-82ba-5f32613e5def-kube-api-access-tcpvc\") pod \"05adb50f-e57d-49ad-82ba-5f32613e5def\" (UID: \"05adb50f-e57d-49ad-82ba-5f32613e5def\") "
May 29 02:37:13.669425 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.669399    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05adb50f-e57d-49ad-82ba-5f32613e5def-kube-api-access-tcpvc" (OuterVolumeSpecName: "kube-api-access-tcpvc") pod "05adb50f-e57d-49ad-82ba-5f32613e5def" (UID: "05adb50f-e57d-49ad-82ba-5f32613e5def"). InnerVolumeSpecName "kube-api-access-tcpvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 02:37:13.768580 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.768550    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcpvc\" (UniqueName: \"kubernetes.io/projected/05adb50f-e57d-49ad-82ba-5f32613e5def-kube-api-access-tcpvc\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:37:13.798031 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.798009    2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-z2rjz"
May 29 02:37:13.970264 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.970189    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z92rp\" (UniqueName: \"kubernetes.io/projected/a7a0d744-912d-444f-9646-042961850bcb-kube-api-access-z92rp\") pod \"a7a0d744-912d-444f-9646-042961850bcb\" (UID: \"a7a0d744-912d-444f-9646-042961850bcb\") "
May 29 02:37:13.972322 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:13.972286    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7a0d744-912d-444f-9646-042961850bcb-kube-api-access-z92rp" (OuterVolumeSpecName: "kube-api-access-z92rp") pod "a7a0d744-912d-444f-9646-042961850bcb" (UID: "a7a0d744-912d-444f-9646-042961850bcb"). InnerVolumeSpecName "kube-api-access-z92rp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 02:37:14.070883 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:14.070845    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z92rp\" (UniqueName: \"kubernetes.io/projected/a7a0d744-912d-444f-9646-042961850bcb-kube-api-access-z92rp\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:37:14.554866 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:14.554819    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5db5594495-8qlpd" event={"ID":"ec1ef264-a21e-4066-acc0-a26747b2307a","Type":"ContainerStarted","Data":"2487682a8710991dc62fb9d57500e343cfeff5093cabd3c67970d7b8764c754a"}
May 29 02:37:14.556059 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:14.556033    2553 generic.go:358] "Generic (PLEG): container finished" podID="a7a0d744-912d-444f-9646-042961850bcb" containerID="b9e56a8ea0f9feae80d022f5ba8347fffd250fdaa0af2a2b0bbf5b26a2c4d2f4" exitCode=0
May 29 02:37:14.556201 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:14.556095    2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-z2rjz"
May 29 02:37:14.556201 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:14.556109    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-z2rjz" event={"ID":"a7a0d744-912d-444f-9646-042961850bcb","Type":"ContainerDied","Data":"b9e56a8ea0f9feae80d022f5ba8347fffd250fdaa0af2a2b0bbf5b26a2c4d2f4"}
May 29 02:37:14.556201 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:14.556138    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-z2rjz" event={"ID":"a7a0d744-912d-444f-9646-042961850bcb","Type":"ContainerDied","Data":"4e9437f9fbeb329b8028efb2d541e3a86d45404b99b852fd78e7bb736b54b93b"}
May 29 02:37:14.556201 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:14.556163    2553 scope.go:117] "RemoveContainer" containerID="b9e56a8ea0f9feae80d022f5ba8347fffd250fdaa0af2a2b0bbf5b26a2c4d2f4"
May 29 02:37:14.556393 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:14.556282    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bffcc75dc-7bdxp"
May 29 02:37:14.565487 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:14.565471    2553 scope.go:117] "RemoveContainer" containerID="b9e56a8ea0f9feae80d022f5ba8347fffd250fdaa0af2a2b0bbf5b26a2c4d2f4"
May 29 02:37:14.565817 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:37:14.565794    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9e56a8ea0f9feae80d022f5ba8347fffd250fdaa0af2a2b0bbf5b26a2c4d2f4\": container with ID starting with b9e56a8ea0f9feae80d022f5ba8347fffd250fdaa0af2a2b0bbf5b26a2c4d2f4 not found: ID does not exist" containerID="b9e56a8ea0f9feae80d022f5ba8347fffd250fdaa0af2a2b0bbf5b26a2c4d2f4"
May 29 02:37:14.565915 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:14.565827    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9e56a8ea0f9feae80d022f5ba8347fffd250fdaa0af2a2b0bbf5b26a2c4d2f4"} err="failed to get container status \"b9e56a8ea0f9feae80d022f5ba8347fffd250fdaa0af2a2b0bbf5b26a2c4d2f4\": rpc error: code = NotFound desc = could not find container \"b9e56a8ea0f9feae80d022f5ba8347fffd250fdaa0af2a2b0bbf5b26a2c4d2f4\": container with ID starting with b9e56a8ea0f9feae80d022f5ba8347fffd250fdaa0af2a2b0bbf5b26a2c4d2f4 not found: ID does not exist"
May 29 02:37:14.573158 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:14.573111    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-5db5594495-8qlpd" podStartSLOduration=1.27239076 podStartE2EDuration="1.573098476s" podCreationTimestamp="2026-05-29 02:37:13 +0000 UTC" firstStartedPulling="2026-05-29 02:37:13.543153866 +0000 UTC m=+574.406766376" lastFinishedPulling="2026-05-29 02:37:13.843861581 +0000 UTC m=+574.707474092" observedRunningTime="2026-05-29 02:37:14.570672294 +0000 UTC m=+575.434284825" watchObservedRunningTime="2026-05-29 02:37:14.573098476 +0000 UTC m=+575.436711045"
May 29 02:37:14.588355 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:14.588327    2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-z2rjz"]
May 29 02:37:14.594060 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:14.594030    2553 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-z2rjz"]
May 29 02:37:14.623905 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:14.623873    2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bffcc75dc-7bdxp"]
May 29 02:37:14.627825 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:14.627781    2553 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bffcc75dc-7bdxp"]
May 29 02:37:15.719888 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:15.719857    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05adb50f-e57d-49ad-82ba-5f32613e5def" path="/var/lib/kubelet/pods/05adb50f-e57d-49ad-82ba-5f32613e5def/volumes"
May 29 02:37:15.720261 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:37:15.720082    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7a0d744-912d-444f-9646-042961850bcb" path="/var/lib/kubelet/pods/a7a0d744-912d-444f-9646-042961850bcb/volumes"
May 29 02:38:09.416813 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.416745    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"]
May 29 02:38:09.419269 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.417080    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7a0d744-912d-444f-9646-042961850bcb" containerName="authorino"
May 29 02:38:09.419269 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.417090    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7a0d744-912d-444f-9646-042961850bcb" containerName="authorino"
May 29 02:38:09.419269 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.417143    2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="a7a0d744-912d-444f-9646-042961850bcb" containerName="authorino"
May 29 02:38:09.420241 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.420226    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.423211 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.423192    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
May 29 02:38:09.423344 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.423211    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
May 29 02:38:09.424328 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.424305    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
May 29 02:38:09.424420 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.424366    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qbjgj\""
May 29 02:38:09.433086 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.433068    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"]
May 29 02:38:09.533781 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.533754    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5407b618-5434-4f9f-bc19-bff4a372fefa-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.533910 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.533790    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5407b618-5434-4f9f-bc19-bff4a372fefa-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.533910 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.533816    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5407b618-5434-4f9f-bc19-bff4a372fefa-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.533910 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.533833    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5hgc\" (UniqueName: \"kubernetes.io/projected/5407b618-5434-4f9f-bc19-bff4a372fefa-kube-api-access-v5hgc\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.534019 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.533935    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5407b618-5434-4f9f-bc19-bff4a372fefa-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.534019 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.533963    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5407b618-5434-4f9f-bc19-bff4a372fefa-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.634713 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.634677    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5407b618-5434-4f9f-bc19-bff4a372fefa-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.634713 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.634715    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5407b618-5434-4f9f-bc19-bff4a372fefa-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.634956 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.634760    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5407b618-5434-4f9f-bc19-bff4a372fefa-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.634956 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.634784    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5407b618-5434-4f9f-bc19-bff4a372fefa-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.634956 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.634810    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5407b618-5434-4f9f-bc19-bff4a372fefa-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.634956 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.634832    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v5hgc\" (UniqueName: \"kubernetes.io/projected/5407b618-5434-4f9f-bc19-bff4a372fefa-kube-api-access-v5hgc\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.635184 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.635159    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5407b618-5434-4f9f-bc19-bff4a372fefa-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.635242 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.635208    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5407b618-5434-4f9f-bc19-bff4a372fefa-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.635242 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.635221    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5407b618-5434-4f9f-bc19-bff4a372fefa-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.637092 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.637074    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5407b618-5434-4f9f-bc19-bff4a372fefa-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.637228 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.637210    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5407b618-5434-4f9f-bc19-bff4a372fefa-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.643150 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.643128    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5hgc\" (UniqueName: \"kubernetes.io/projected/5407b618-5434-4f9f-bc19-bff4a372fefa-kube-api-access-v5hgc\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h\" (UID: \"5407b618-5434-4f9f-bc19-bff4a372fefa\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.730145 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.730075    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:09.871205 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:09.871177    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"]
May 29 02:38:09.873466 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:38:09.873439    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5407b618_5434_4f9f_bc19_bff4a372fefa.slice/crio-026064241a6b49fa39c3f3ceacb3684c6bfe41bd2be39be50583d5986c82dd59 WatchSource:0}: Error finding container 026064241a6b49fa39c3f3ceacb3684c6bfe41bd2be39be50583d5986c82dd59: Status 404 returned error can't find the container with id 026064241a6b49fa39c3f3ceacb3684c6bfe41bd2be39be50583d5986c82dd59
May 29 02:38:10.746286 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:10.746243    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h" event={"ID":"5407b618-5434-4f9f-bc19-bff4a372fefa","Type":"ContainerStarted","Data":"026064241a6b49fa39c3f3ceacb3684c6bfe41bd2be39be50583d5986c82dd59"}
May 29 02:38:14.761601 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:14.761556    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h" event={"ID":"5407b618-5434-4f9f-bc19-bff4a372fefa","Type":"ContainerStarted","Data":"f65b8d889f0dc09d8ea236c849d8b1b3441f85b3add45d297ea8bc2e37a8d5c9"}
May 29 02:38:15.766627 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:15.766588    2553 generic.go:358] "Generic (PLEG): container finished" podID="5407b618-5434-4f9f-bc19-bff4a372fefa" containerID="f65b8d889f0dc09d8ea236c849d8b1b3441f85b3add45d297ea8bc2e37a8d5c9" exitCode=0
May 29 02:38:15.767120 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:15.766693    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h" event={"ID":"5407b618-5434-4f9f-bc19-bff4a372fefa","Type":"ContainerDied","Data":"f65b8d889f0dc09d8ea236c849d8b1b3441f85b3add45d297ea8bc2e37a8d5c9"}
May 29 02:38:17.775397 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:17.775360    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h" event={"ID":"5407b618-5434-4f9f-bc19-bff4a372fefa","Type":"ContainerStarted","Data":"9d48f2c9c1083a343cb08344a92dba51104626d24dad1dec18e7ba3d1acf2b60"}
May 29 02:38:17.796265 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:17.796220    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h" podStartSLOduration=1.8378266650000001 podStartE2EDuration="8.796204705s" podCreationTimestamp="2026-05-29 02:38:09 +0000 UTC" firstStartedPulling="2026-05-29 02:38:09.875355204 +0000 UTC m=+630.738967714" lastFinishedPulling="2026-05-29 02:38:16.83373324 +0000 UTC m=+637.697345754" observedRunningTime="2026-05-29 02:38:17.794704019 +0000 UTC m=+638.658316551" watchObservedRunningTime="2026-05-29 02:38:17.796204705 +0000 UTC m=+638.659817236"
May 29 02:38:19.731177 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:19.731137    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:19.731540 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:19.731195    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:19.743541 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:19.743516    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:19.794467 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:19.794437    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h"
May 29 02:38:57.505853 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.505817    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"]
May 29 02:38:57.508363 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.508346    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.511032 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.511013    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
May 29 02:38:57.518626 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.518601    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"]
May 29 02:38:57.670418 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.670374    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdbds\" (UniqueName: \"kubernetes.io/projected/991c1050-2d60-47e2-9724-e303e5da6329-kube-api-access-fdbds\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.670615 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.670466    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/991c1050-2d60-47e2-9724-e303e5da6329-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.670615 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.670512    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/991c1050-2d60-47e2-9724-e303e5da6329-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.670752 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.670618    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/991c1050-2d60-47e2-9724-e303e5da6329-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.670752 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.670729    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/991c1050-2d60-47e2-9724-e303e5da6329-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.670834 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.670788    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/991c1050-2d60-47e2-9724-e303e5da6329-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.771473 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.771442    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/991c1050-2d60-47e2-9724-e303e5da6329-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.771687 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.771488    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/991c1050-2d60-47e2-9724-e303e5da6329-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.771687 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.771622    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/991c1050-2d60-47e2-9724-e303e5da6329-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.771813 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.771689    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fdbds\" (UniqueName: \"kubernetes.io/projected/991c1050-2d60-47e2-9724-e303e5da6329-kube-api-access-fdbds\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.771813 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.771750    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/991c1050-2d60-47e2-9724-e303e5da6329-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.771813 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.771782    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/991c1050-2d60-47e2-9724-e303e5da6329-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.771964 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.771821    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/991c1050-2d60-47e2-9724-e303e5da6329-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.771964 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.771919    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/991c1050-2d60-47e2-9724-e303e5da6329-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.772157 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.772133    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/991c1050-2d60-47e2-9724-e303e5da6329-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.774114 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.774090    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/991c1050-2d60-47e2-9724-e303e5da6329-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.774345 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.774329    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/991c1050-2d60-47e2-9724-e303e5da6329-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.779801 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.779780    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdbds\" (UniqueName: \"kubernetes.io/projected/991c1050-2d60-47e2-9724-e303e5da6329-kube-api-access-fdbds\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k\" (UID: \"991c1050-2d60-47e2-9724-e303e5da6329\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.819878 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.819843    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:38:57.953826 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.953801    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"]
May 29 02:38:57.955500 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:38:57.955470    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod991c1050_2d60_47e2_9724_e303e5da6329.slice/crio-390e64ae1550b6d30f3f1e01ed12eba4123bf99b451b3e432245f27e559d8496 WatchSource:0}: Error finding container 390e64ae1550b6d30f3f1e01ed12eba4123bf99b451b3e432245f27e559d8496: Status 404 returned error can't find the container with id 390e64ae1550b6d30f3f1e01ed12eba4123bf99b451b3e432245f27e559d8496
May 29 02:38:57.957273 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:57.957251    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 02:38:58.919094 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:58.919045    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k" event={"ID":"991c1050-2d60-47e2-9724-e303e5da6329","Type":"ContainerStarted","Data":"0cf93a47c2c317538b82be1503d3d27cc8ec114eb579cffd44e25d01bd36d9a5"}
May 29 02:38:58.919094 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:58.919089    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k" event={"ID":"991c1050-2d60-47e2-9724-e303e5da6329","Type":"ContainerStarted","Data":"390e64ae1550b6d30f3f1e01ed12eba4123bf99b451b3e432245f27e559d8496"}
May 29 02:38:59.923873 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:59.923839    2553 generic.go:358] "Generic (PLEG): container finished" podID="991c1050-2d60-47e2-9724-e303e5da6329" containerID="0cf93a47c2c317538b82be1503d3d27cc8ec114eb579cffd44e25d01bd36d9a5" exitCode=0
May 29 02:38:59.924251 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:38:59.923934    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k" event={"ID":"991c1050-2d60-47e2-9724-e303e5da6329","Type":"ContainerDied","Data":"0cf93a47c2c317538b82be1503d3d27cc8ec114eb579cffd44e25d01bd36d9a5"}
May 29 02:39:00.929223 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:00.929187    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k" event={"ID":"991c1050-2d60-47e2-9724-e303e5da6329","Type":"ContainerStarted","Data":"f71b45a57216145354e7cebaedfc43497a1b225a2d0ec361272106a989735276"}
May 29 02:39:00.954249 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:00.954192    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k" podStartSLOduration=3.8013747540000002 podStartE2EDuration="3.954177754s" podCreationTimestamp="2026-05-29 02:38:57 +0000 UTC" firstStartedPulling="2026-05-29 02:38:59.924630916 +0000 UTC m=+680.788243432" lastFinishedPulling="2026-05-29 02:39:00.07743392 +0000 UTC m=+680.941046432" observedRunningTime="2026-05-29 02:39:00.952192248 +0000 UTC m=+681.815804779" watchObservedRunningTime="2026-05-29 02:39:00.954177754 +0000 UTC m=+681.817790284"
May 29 02:39:07.820461 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:07.820411    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:39:07.820461 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:07.820466    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:39:07.832709 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:07.832643    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:39:07.963995 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:07.963962    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k"
May 29 02:39:15.608798 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.608768    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"]
May 29 02:39:15.611892 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.611872    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.615015 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.614998    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
May 29 02:39:15.625886 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.625865    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"]
May 29 02:39:15.708424 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.708396    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w9gk\" (UniqueName: \"kubernetes.io/projected/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-kube-api-access-7w9gk\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.708562 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.708467    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.708562 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.708492    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.708562 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.708513    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.708562 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.708533    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.708734 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.708572    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.809517 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.809475    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.809720 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.809531    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.809720 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.809573    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.809720 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.809706    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.809887 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.809755    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.809887 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.809826    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7w9gk\" (UniqueName: \"kubernetes.io/projected/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-kube-api-access-7w9gk\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.809992 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.809972    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.810435 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.810410    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.810731 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.810436    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.812441 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.812419    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.812552 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.812528    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.818880 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.818858    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w9gk\" (UniqueName: \"kubernetes.io/projected/c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a-kube-api-access-7w9gk\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6\" (UID: \"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:15.922152 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:15.922046    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:16.055150 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:16.055125    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"]
May 29 02:39:16.056720 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:39:16.056690    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1af1fcc_e88d_4bdb_9eb8_66914e24ea0a.slice/crio-382c5e135e9e077f4955183ee1294ee05daf5c9471e373db83fdd33abe234ae6 WatchSource:0}: Error finding container 382c5e135e9e077f4955183ee1294ee05daf5c9471e373db83fdd33abe234ae6: Status 404 returned error can't find the container with id 382c5e135e9e077f4955183ee1294ee05daf5c9471e373db83fdd33abe234ae6
May 29 02:39:16.986314 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:16.986273    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6" event={"ID":"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a","Type":"ContainerStarted","Data":"ded5c6b0bf8c3879b05626db85f5d1635f3520f3a8a29d8aaeabb13ff2baab2e"}
May 29 02:39:16.986867 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:16.986322    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6" event={"ID":"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a","Type":"ContainerStarted","Data":"382c5e135e9e077f4955183ee1294ee05daf5c9471e373db83fdd33abe234ae6"}
May 29 02:39:17.991271 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:17.991236    2553 generic.go:358] "Generic (PLEG): container finished" podID="c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a" containerID="ded5c6b0bf8c3879b05626db85f5d1635f3520f3a8a29d8aaeabb13ff2baab2e" exitCode=0
May 29 02:39:17.991673 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:17.991329    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6" event={"ID":"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a","Type":"ContainerDied","Data":"ded5c6b0bf8c3879b05626db85f5d1635f3520f3a8a29d8aaeabb13ff2baab2e"}
May 29 02:39:18.995851 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:18.995771    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6" event={"ID":"c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a","Type":"ContainerStarted","Data":"b488c51dcf4ada8e98984d5fdba5fee543426e601ec6b1f8fb4d15f446ec9c44"}
May 29 02:39:19.018128 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:19.018086    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6" podStartSLOduration=3.866682784 podStartE2EDuration="4.018070394s" podCreationTimestamp="2026-05-29 02:39:15 +0000 UTC" firstStartedPulling="2026-05-29 02:39:17.991837703 +0000 UTC m=+698.855450213" lastFinishedPulling="2026-05-29 02:39:18.143225311 +0000 UTC m=+699.006837823" observedRunningTime="2026-05-29 02:39:19.016838436 +0000 UTC m=+699.880450969" watchObservedRunningTime="2026-05-29 02:39:19.018070394 +0000 UTC m=+699.881682926"
May 29 02:39:25.923088 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:25.923004    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:25.923444 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:25.923152    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:25.935542 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:25.935513    2553 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:39:26.031235 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:39:26.031210    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6"
May 29 02:40:50.704290 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:50.704253    2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5db5594495-8qlpd"]
May 29 02:40:50.704939 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:50.704479    2553 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-5db5594495-8qlpd" podUID="ec1ef264-a21e-4066-acc0-a26747b2307a" containerName="authorino" containerID="cri-o://2487682a8710991dc62fb9d57500e343cfeff5093cabd3c67970d7b8764c754a" gracePeriod=30
May 29 02:40:50.947592 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:50.947568    2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5db5594495-8qlpd"
May 29 02:40:50.972745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:50.972565    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec1ef264-a21e-4066-acc0-a26747b2307a-tls-cert\") pod \"ec1ef264-a21e-4066-acc0-a26747b2307a\" (UID: \"ec1ef264-a21e-4066-acc0-a26747b2307a\") "
May 29 02:40:50.972745 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:50.972705    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kbql\" (UniqueName: \"kubernetes.io/projected/ec1ef264-a21e-4066-acc0-a26747b2307a-kube-api-access-9kbql\") pod \"ec1ef264-a21e-4066-acc0-a26747b2307a\" (UID: \"ec1ef264-a21e-4066-acc0-a26747b2307a\") "
May 29 02:40:50.975446 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:50.975413    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec1ef264-a21e-4066-acc0-a26747b2307a-kube-api-access-9kbql" (OuterVolumeSpecName: "kube-api-access-9kbql") pod "ec1ef264-a21e-4066-acc0-a26747b2307a" (UID: "ec1ef264-a21e-4066-acc0-a26747b2307a"). InnerVolumeSpecName "kube-api-access-9kbql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 02:40:50.985249 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:50.985221    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec1ef264-a21e-4066-acc0-a26747b2307a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ec1ef264-a21e-4066-acc0-a26747b2307a" (UID: "ec1ef264-a21e-4066-acc0-a26747b2307a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 02:40:51.073545 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:51.073505    2553 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ec1ef264-a21e-4066-acc0-a26747b2307a-tls-cert\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:40:51.073545 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:51.073539    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kbql\" (UniqueName: \"kubernetes.io/projected/ec1ef264-a21e-4066-acc0-a26747b2307a-kube-api-access-9kbql\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:40:51.312772 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:51.312736    2553 generic.go:358] "Generic (PLEG): container finished" podID="ec1ef264-a21e-4066-acc0-a26747b2307a" containerID="2487682a8710991dc62fb9d57500e343cfeff5093cabd3c67970d7b8764c754a" exitCode=0
May 29 02:40:51.312956 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:51.312791    2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5db5594495-8qlpd"
May 29 02:40:51.312956 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:51.312826    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5db5594495-8qlpd" event={"ID":"ec1ef264-a21e-4066-acc0-a26747b2307a","Type":"ContainerDied","Data":"2487682a8710991dc62fb9d57500e343cfeff5093cabd3c67970d7b8764c754a"}
May 29 02:40:51.312956 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:51.312867    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5db5594495-8qlpd" event={"ID":"ec1ef264-a21e-4066-acc0-a26747b2307a","Type":"ContainerDied","Data":"bdb192ede64d0f004db001da7ec13a3887ba5191f083053189e6c4f4d7fb0908"}
May 29 02:40:51.312956 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:51.312887    2553 scope.go:117] "RemoveContainer" containerID="2487682a8710991dc62fb9d57500e343cfeff5093cabd3c67970d7b8764c754a"
May 29 02:40:51.321505 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:51.321479    2553 scope.go:117] "RemoveContainer" containerID="2487682a8710991dc62fb9d57500e343cfeff5093cabd3c67970d7b8764c754a"
May 29 02:40:51.321793 ip-10-0-133-245 kubenswrapper[2553]: E0529 02:40:51.321770    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2487682a8710991dc62fb9d57500e343cfeff5093cabd3c67970d7b8764c754a\": container with ID starting with 2487682a8710991dc62fb9d57500e343cfeff5093cabd3c67970d7b8764c754a not found: ID does not exist" containerID="2487682a8710991dc62fb9d57500e343cfeff5093cabd3c67970d7b8764c754a"
May 29 02:40:51.321852 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:51.321803    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2487682a8710991dc62fb9d57500e343cfeff5093cabd3c67970d7b8764c754a"} err="failed to get container status \"2487682a8710991dc62fb9d57500e343cfeff5093cabd3c67970d7b8764c754a\": rpc error: code = NotFound desc = could not find container \"2487682a8710991dc62fb9d57500e343cfeff5093cabd3c67970d7b8764c754a\": container with ID starting with 2487682a8710991dc62fb9d57500e343cfeff5093cabd3c67970d7b8764c754a not found: ID does not exist"
May 29 02:40:51.335417 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:51.335388    2553 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5db5594495-8qlpd"]
May 29 02:40:51.337124 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:51.337099    2553 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5db5594495-8qlpd"]
May 29 02:40:51.720711 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:40:51.720603    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec1ef264-a21e-4066-acc0-a26747b2307a" path="/var/lib/kubelet/pods/ec1ef264-a21e-4066-acc0-a26747b2307a/volumes"
May 29 02:45:00.141909 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:00.141872    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29667045-gxh6s"]
May 29 02:45:00.143289 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:00.142203    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec1ef264-a21e-4066-acc0-a26747b2307a" containerName="authorino"
May 29 02:45:00.143289 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:00.142213    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec1ef264-a21e-4066-acc0-a26747b2307a" containerName="authorino"
May 29 02:45:00.143289 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:00.142293    2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="ec1ef264-a21e-4066-acc0-a26747b2307a" containerName="authorino"
May 29 02:45:00.144133 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:00.144110    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29667045-gxh6s"
May 29 02:45:00.146562 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:00.146544    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-z2m7m\""
May 29 02:45:00.192542 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:00.192504    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29667045-gxh6s"]
May 29 02:45:00.237771 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:00.237722    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cxdn\" (UniqueName: \"kubernetes.io/projected/7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3-kube-api-access-4cxdn\") pod \"maas-api-key-cleanup-29667045-gxh6s\" (UID: \"7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3\") " pod="opendatahub/maas-api-key-cleanup-29667045-gxh6s"
May 29 02:45:00.338465 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:00.338429    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4cxdn\" (UniqueName: \"kubernetes.io/projected/7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3-kube-api-access-4cxdn\") pod \"maas-api-key-cleanup-29667045-gxh6s\" (UID: \"7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3\") " pod="opendatahub/maas-api-key-cleanup-29667045-gxh6s"
May 29 02:45:00.347077 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:00.347044    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cxdn\" (UniqueName: \"kubernetes.io/projected/7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3-kube-api-access-4cxdn\") pod \"maas-api-key-cleanup-29667045-gxh6s\" (UID: \"7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3\") " pod="opendatahub/maas-api-key-cleanup-29667045-gxh6s"
May 29 02:45:00.453911 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:00.453812    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29667045-gxh6s"
May 29 02:45:00.579489 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:00.579465    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29667045-gxh6s"]
May 29 02:45:00.581513 ip-10-0-133-245 kubenswrapper[2553]: W0529 02:45:00.581485    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bab2ee4_f3f9_4269_b3c9_12a8c6674aa3.slice/crio-d0b95b899cabbf29505e8cbdc9cf15884a39c897e02dc18f006abd0408de76e3 WatchSource:0}: Error finding container d0b95b899cabbf29505e8cbdc9cf15884a39c897e02dc18f006abd0408de76e3: Status 404 returned error can't find the container with id d0b95b899cabbf29505e8cbdc9cf15884a39c897e02dc18f006abd0408de76e3
May 29 02:45:00.583221 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:00.583200    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 02:45:01.150730 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:01.150691    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29667045-gxh6s" event={"ID":"7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3","Type":"ContainerStarted","Data":"d0b95b899cabbf29505e8cbdc9cf15884a39c897e02dc18f006abd0408de76e3"}
May 29 02:45:04.164170 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:04.164133    2553 generic.go:358] "Generic (PLEG): container finished" podID="7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3" containerID="ce2ad446142a58ba5cf4ff0d4aca388906f6245c3b8cda32a9b4c55320e30150" exitCode=0
May 29 02:45:04.164513 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:04.164195    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29667045-gxh6s" event={"ID":"7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3","Type":"ContainerDied","Data":"ce2ad446142a58ba5cf4ff0d4aca388906f6245c3b8cda32a9b4c55320e30150"}
May 29 02:45:05.296412 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:05.296389    2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29667045-gxh6s"
May 29 02:45:05.393040 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:05.393009    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cxdn\" (UniqueName: \"kubernetes.io/projected/7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3-kube-api-access-4cxdn\") pod \"7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3\" (UID: \"7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3\") "
May 29 02:45:05.395221 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:05.395201    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3-kube-api-access-4cxdn" (OuterVolumeSpecName: "kube-api-access-4cxdn") pod "7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3" (UID: "7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3"). InnerVolumeSpecName "kube-api-access-4cxdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 02:45:05.494440 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:05.494367    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cxdn\" (UniqueName: \"kubernetes.io/projected/7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3-kube-api-access-4cxdn\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 02:45:06.171176 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:06.171146    2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29667045-gxh6s"
May 29 02:45:06.171176 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:06.171165    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29667045-gxh6s" event={"ID":"7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3","Type":"ContainerDied","Data":"d0b95b899cabbf29505e8cbdc9cf15884a39c897e02dc18f006abd0408de76e3"}
May 29 02:45:06.171409 ip-10-0-133-245 kubenswrapper[2553]: I0529 02:45:06.171193    2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0b95b899cabbf29505e8cbdc9cf15884a39c897e02dc18f006abd0408de76e3"
May 29 03:00:00.148852 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:00.148812    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29667060-4nk6k"]
May 29 03:00:00.150106 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:00.149165    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3" containerName="cleanup"
May 29 03:00:00.150106 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:00.149176    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3" containerName="cleanup"
May 29 03:00:00.150106 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:00.149240    2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3" containerName="cleanup"
May 29 03:00:00.151555 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:00.151538    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29667060-4nk6k"
May 29 03:00:00.154996 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:00.154974    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-z2m7m\""
May 29 03:00:00.182467 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:00.182435    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nrnx\" (UniqueName: \"kubernetes.io/projected/86ac4a71-d8bc-4e0b-b876-da3aaf66c845-kube-api-access-7nrnx\") pod \"maas-api-key-cleanup-29667060-4nk6k\" (UID: \"86ac4a71-d8bc-4e0b-b876-da3aaf66c845\") " pod="opendatahub/maas-api-key-cleanup-29667060-4nk6k"
May 29 03:00:00.231419 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:00.231388    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29667060-4nk6k"]
May 29 03:00:00.283125 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:00.283092    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7nrnx\" (UniqueName: \"kubernetes.io/projected/86ac4a71-d8bc-4e0b-b876-da3aaf66c845-kube-api-access-7nrnx\") pod \"maas-api-key-cleanup-29667060-4nk6k\" (UID: \"86ac4a71-d8bc-4e0b-b876-da3aaf66c845\") " pod="opendatahub/maas-api-key-cleanup-29667060-4nk6k"
May 29 03:00:00.293582 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:00.293552    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nrnx\" (UniqueName: \"kubernetes.io/projected/86ac4a71-d8bc-4e0b-b876-da3aaf66c845-kube-api-access-7nrnx\") pod \"maas-api-key-cleanup-29667060-4nk6k\" (UID: \"86ac4a71-d8bc-4e0b-b876-da3aaf66c845\") " pod="opendatahub/maas-api-key-cleanup-29667060-4nk6k"
May 29 03:00:00.461863 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:00.461783    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29667060-4nk6k"
May 29 03:00:00.591086 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:00.591058    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29667060-4nk6k"]
May 29 03:00:00.594041 ip-10-0-133-245 kubenswrapper[2553]: W0529 03:00:00.594012    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86ac4a71_d8bc_4e0b_b876_da3aaf66c845.slice/crio-3239b6a5b64399e155c61648ab535171ebb2353f3b3a7c9a91f382ded9bc1fe2 WatchSource:0}: Error finding container 3239b6a5b64399e155c61648ab535171ebb2353f3b3a7c9a91f382ded9bc1fe2: Status 404 returned error can't find the container with id 3239b6a5b64399e155c61648ab535171ebb2353f3b3a7c9a91f382ded9bc1fe2
May 29 03:00:00.596263 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:00.596247    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 03:00:01.242597 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:01.242519    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29667060-4nk6k" event={"ID":"86ac4a71-d8bc-4e0b-b876-da3aaf66c845","Type":"ContainerStarted","Data":"16a58f358a806a511cd10e765dde00cce7cf4aad2478f6b1cf5f712a31fb70fe"}
May 29 03:00:01.242597 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:01.242554    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29667060-4nk6k" event={"ID":"86ac4a71-d8bc-4e0b-b876-da3aaf66c845","Type":"ContainerStarted","Data":"3239b6a5b64399e155c61648ab535171ebb2353f3b3a7c9a91f382ded9bc1fe2"}
May 29 03:00:01.261723 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:01.261630    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29667060-4nk6k" podStartSLOduration=1.261611665 podStartE2EDuration="1.261611665s" podCreationTimestamp="2026-05-29 03:00:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 03:00:01.259241578 +0000 UTC m=+1942.122854111" watchObservedRunningTime="2026-05-29 03:00:01.261611665 +0000 UTC m=+1942.125224200"
May 29 03:00:02.247997 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:02.247963    2553 generic.go:358] "Generic (PLEG): container finished" podID="86ac4a71-d8bc-4e0b-b876-da3aaf66c845" containerID="16a58f358a806a511cd10e765dde00cce7cf4aad2478f6b1cf5f712a31fb70fe" exitCode=0
May 29 03:00:02.248440 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:02.248032    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29667060-4nk6k" event={"ID":"86ac4a71-d8bc-4e0b-b876-da3aaf66c845","Type":"ContainerDied","Data":"16a58f358a806a511cd10e765dde00cce7cf4aad2478f6b1cf5f712a31fb70fe"}
May 29 03:00:03.379945 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:03.379917    2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29667060-4nk6k"
May 29 03:00:03.515147 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:03.515119    2553 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nrnx\" (UniqueName: \"kubernetes.io/projected/86ac4a71-d8bc-4e0b-b876-da3aaf66c845-kube-api-access-7nrnx\") pod \"86ac4a71-d8bc-4e0b-b876-da3aaf66c845\" (UID: \"86ac4a71-d8bc-4e0b-b876-da3aaf66c845\") "
May 29 03:00:03.517365 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:03.517338    2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86ac4a71-d8bc-4e0b-b876-da3aaf66c845-kube-api-access-7nrnx" (OuterVolumeSpecName: "kube-api-access-7nrnx") pod "86ac4a71-d8bc-4e0b-b876-da3aaf66c845" (UID: "86ac4a71-d8bc-4e0b-b876-da3aaf66c845"). InnerVolumeSpecName "kube-api-access-7nrnx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 03:00:03.616078 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:03.616042    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nrnx\" (UniqueName: \"kubernetes.io/projected/86ac4a71-d8bc-4e0b-b876-da3aaf66c845-kube-api-access-7nrnx\") on node \"ip-10-0-133-245.ec2.internal\" DevicePath \"\""
May 29 03:00:04.256848 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:04.256811    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29667060-4nk6k" event={"ID":"86ac4a71-d8bc-4e0b-b876-da3aaf66c845","Type":"ContainerDied","Data":"3239b6a5b64399e155c61648ab535171ebb2353f3b3a7c9a91f382ded9bc1fe2"}
May 29 03:00:04.256848 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:04.256850    2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3239b6a5b64399e155c61648ab535171ebb2353f3b3a7c9a91f382ded9bc1fe2"
May 29 03:00:04.256848 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:00:04.256859    2553 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29667060-4nk6k"
May 29 03:03:51.872612 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:03:51.872534    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-55b468f665-9jlkr_deaaf730-c1d8-44c6-811a-e72ce67ed1c1/nginx/0.log"
May 29 03:03:52.452355 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:03:52.452328    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29667045-gxh6s_7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3/cleanup/0.log"
May 29 03:03:52.562457 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:03:52.562432    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29667060-4nk6k_86ac4a71-d8bc-4e0b-b876-da3aaf66c845/cleanup/0.log"
May 29 03:03:53.155044 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:03:53.155015    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-784f8c8bfd-ggzqr_2974d6c0-41cb-4276-9890-a3b55af1f299/copy-manifests/0.log"
May 29 03:03:53.162132 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:03:53.162106    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-784f8c8bfd-ggzqr_2974d6c0-41cb-4276-9890-a3b55af1f299/manager/0.log"
May 29 03:03:54.757962 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:03:54.757930    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-spg8s_d1135c53-67c0-448d-8413-d79be85b5c8a/manager/0.log"
May 29 03:03:55.095479 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:03:55.095444    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2p2j2_92b38dd5-c056-4fc4-a772-46073f270636/registry-server/0.log"
May 29 03:03:55.452625 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:03:55.452536    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-5n9ff_71924069-77b1-4a35-88cf-5e5207d98493/manager/0.log"
May 29 03:03:55.798852 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:03:55.798812    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8_2120d8c1-9172-44cc-af86-c23c1ae2580e/istio-proxy/0.log"
May 29 03:03:56.247616 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:03:56.247536    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-s8rdr_3efd16c1-2a11-4eb6-a42f-e7326eea722a/istio-proxy/0.log"
May 29 03:03:56.798231 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:03:56.798200    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h_5407b618-5434-4f9f-bc19-bff4a372fefa/main/0.log"
May 29 03:03:56.805394 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:03:56.805366    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-jpd2h_5407b618-5434-4f9f-bc19-bff4a372fefa/storage-initializer/0.log"
May 29 03:03:57.154901 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:03:57.154821    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k_991c1050-2d60-47e2-9724-e303e5da6329/storage-initializer/0.log"
May 29 03:03:57.167333 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:03:57.167304    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9xpk9k_991c1050-2d60-47e2-9724-e303e5da6329/main/0.log"
May 29 03:03:57.276275 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:03:57.276245    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6_c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a/storage-initializer/0.log"
May 29 03:03:57.286064 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:03:57.286036    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-92tw6_c1af1fcc-e88d-4bdb-9eb8-66914e24ea0a/main/0.log"
May 29 03:04:04.566311 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:04.566284    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fbhfk_6efc3fe2-975d-4036-b764-dfa0eed1c83c/global-pull-secret-syncer/0.log"
May 29 03:04:04.685481 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:04.685445    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wvm6p_a1da5c79-fc3a-4c1e-b30b-44de60231e7e/konnectivity-agent/0.log"
May 29 03:04:04.762565 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:04.762530    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-245.ec2.internal_f2cb1de6875396d67d2b36a572294b25/haproxy/0.log"
May 29 03:04:09.014758 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:09.014730    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-spg8s_d1135c53-67c0-448d-8413-d79be85b5c8a/manager/0.log"
May 29 03:04:09.119645 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:09.119612    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2p2j2_92b38dd5-c056-4fc4-a772-46073f270636/registry-server/0.log"
May 29 03:04:09.293459 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:09.293381    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-5n9ff_71924069-77b1-4a35-88cf-5e5207d98493/manager/0.log"
May 29 03:04:10.880281 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:10.880252    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5c96c6856c-rrhjm_7c206e11-3d41-48bb-b0f7-d297670d7a71/metrics-server/0.log"
May 29 03:04:11.101739 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.101708    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pstps_34567433-e80a-4963-a6a9-7da83e82115e/node-exporter/0.log"
May 29 03:04:11.125396 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.125371    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pstps_34567433-e80a-4963-a6a9-7da83e82115e/kube-rbac-proxy/0.log"
May 29 03:04:11.146554 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.146529    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pstps_34567433-e80a-4963-a6a9-7da83e82115e/init-textfile/0.log"
May 29 03:04:11.340450 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.340422    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b/prometheus/0.log"
May 29 03:04:11.368079 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.368052    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b/config-reloader/0.log"
May 29 03:04:11.398880 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.398856    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b/thanos-sidecar/0.log"
May 29 03:04:11.434007 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.433987    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b/kube-rbac-proxy-web/0.log"
May 29 03:04:11.466418 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.466396    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b/kube-rbac-proxy/0.log"
May 29 03:04:11.492805 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.492780    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b/kube-rbac-proxy-thanos/0.log"
May 29 03:04:11.517669 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.517613    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_27f4fbf0-b29d-4bd7-83b9-78f9bd962a3b/init-config-reloader/0.log"
May 29 03:04:11.643377 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.643299    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76c88799db-94gsb_4406c398-9433-4a19-a85f-91bbefd9c3b4/telemeter-client/0.log"
May 29 03:04:11.670036 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.670003    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76c88799db-94gsb_4406c398-9433-4a19-a85f-91bbefd9c3b4/reload/0.log"
May 29 03:04:11.693572 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.693546    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76c88799db-94gsb_4406c398-9433-4a19-a85f-91bbefd9c3b4/kube-rbac-proxy/0.log"
May 29 03:04:11.731226 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.731200    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d646c6b65-gkfgm_477f3c90-0808-46bd-a908-4837260e6964/thanos-query/0.log"
May 29 03:04:11.752618 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.752579    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d646c6b65-gkfgm_477f3c90-0808-46bd-a908-4837260e6964/kube-rbac-proxy-web/0.log"
May 29 03:04:11.777299 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.777270    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d646c6b65-gkfgm_477f3c90-0808-46bd-a908-4837260e6964/kube-rbac-proxy/0.log"
May 29 03:04:11.799439 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.799410    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d646c6b65-gkfgm_477f3c90-0808-46bd-a908-4837260e6964/prom-label-proxy/0.log"
May 29 03:04:11.822194 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.822164    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d646c6b65-gkfgm_477f3c90-0808-46bd-a908-4837260e6964/kube-rbac-proxy-rules/0.log"
May 29 03:04:11.851094 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:11.851061    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d646c6b65-gkfgm_477f3c90-0808-46bd-a908-4837260e6964/kube-rbac-proxy-metrics/0.log"
May 29 03:04:15.268544 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.268509    2553 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"]
May 29 03:04:15.268941 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.268866    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86ac4a71-d8bc-4e0b-b876-da3aaf66c845" containerName="cleanup"
May 29 03:04:15.268941 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.268878    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="86ac4a71-d8bc-4e0b-b876-da3aaf66c845" containerName="cleanup"
May 29 03:04:15.269011 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.268949    2553 memory_manager.go:356] "RemoveStaleState removing state" podUID="86ac4a71-d8bc-4e0b-b876-da3aaf66c845" containerName="cleanup"
May 29 03:04:15.271022 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.271005    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.273600 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.273575    2553 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-btflc\"/\"default-dockercfg-md4gp\""
May 29 03:04:15.273749 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.273576    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-btflc\"/\"openshift-service-ca.crt\""
May 29 03:04:15.274536 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.274519    2553 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-btflc\"/\"kube-root-ca.crt\""
May 29 03:04:15.283901 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.283882    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"]
May 29 03:04:15.376750 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.376722    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nxhn\" (UniqueName: \"kubernetes.io/projected/ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da-kube-api-access-7nxhn\") pod \"perf-node-gather-daemonset-pc8tc\" (UID: \"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da\") " pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.376864 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.376766    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da-proc\") pod \"perf-node-gather-daemonset-pc8tc\" (UID: \"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da\") " pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.376864 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.376793    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da-podres\") pod \"perf-node-gather-daemonset-pc8tc\" (UID: \"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da\") " pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.376951 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.376858    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da-sys\") pod \"perf-node-gather-daemonset-pc8tc\" (UID: \"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da\") " pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.376951 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.376937    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da-lib-modules\") pod \"perf-node-gather-daemonset-pc8tc\" (UID: \"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da\") " pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.477622 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.477590    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da-lib-modules\") pod \"perf-node-gather-daemonset-pc8tc\" (UID: \"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da\") " pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.477787 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.477633    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7nxhn\" (UniqueName: \"kubernetes.io/projected/ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da-kube-api-access-7nxhn\") pod \"perf-node-gather-daemonset-pc8tc\" (UID: \"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da\") " pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.477787 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.477693    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da-proc\") pod \"perf-node-gather-daemonset-pc8tc\" (UID: \"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da\") " pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.477787 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.477711    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da-podres\") pod \"perf-node-gather-daemonset-pc8tc\" (UID: \"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da\") " pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.477787 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.477751    2553 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da-sys\") pod \"perf-node-gather-daemonset-pc8tc\" (UID: \"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da\") " pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.477942 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.477783    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da-lib-modules\") pod \"perf-node-gather-daemonset-pc8tc\" (UID: \"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da\") " pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.477942 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.477810    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da-proc\") pod \"perf-node-gather-daemonset-pc8tc\" (UID: \"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da\") " pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.477942 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.477840    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da-sys\") pod \"perf-node-gather-daemonset-pc8tc\" (UID: \"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da\") " pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.477942 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.477853    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da-podres\") pod \"perf-node-gather-daemonset-pc8tc\" (UID: \"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da\") " pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.485640 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.485618    2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nxhn\" (UniqueName: \"kubernetes.io/projected/ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da-kube-api-access-7nxhn\") pod \"perf-node-gather-daemonset-pc8tc\" (UID: \"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da\") " pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.544062 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.544005    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-d4shx_21d4bc93-ef47-4e84-b0ea-4c77774456ea/dns/0.log"
May 29 03:04:15.573734 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.573709    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-d4shx_21d4bc93-ef47-4e84-b0ea-4c77774456ea/kube-rbac-proxy/0.log"
May 29 03:04:15.580991 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.580968    2553 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:15.710525 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.710495    2553 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"]
May 29 03:04:15.713602 ip-10-0-133-245 kubenswrapper[2553]: W0529 03:04:15.713564    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podccc5a5b4_2853_42b4_8ebe_fbac7c7ea8da.slice/crio-ead85c989b9907d8f2fbbad17cfb0e5f4ca7a63e0d5e9aea453b72f3cd56e62c WatchSource:0}: Error finding container ead85c989b9907d8f2fbbad17cfb0e5f4ca7a63e0d5e9aea453b72f3cd56e62c: Status 404 returned error can't find the container with id ead85c989b9907d8f2fbbad17cfb0e5f4ca7a63e0d5e9aea453b72f3cd56e62c
May 29 03:04:15.725442 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:15.725426    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w7gt7_d90c6dde-1b8b-4aba-9d9e-5c11f6e47dd1/dns-node-resolver/0.log"
May 29 03:04:16.127532 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:16.127431    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc" event={"ID":"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da","Type":"ContainerStarted","Data":"cb18f00362b54b7636d14bd7cdd763e2a890a44af4ebd7e251d4e3d8a4db6e59"}
May 29 03:04:16.127532 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:16.127470    2553 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc" event={"ID":"ccc5a5b4-2853-42b4-8ebe-fbac7c7ea8da","Type":"ContainerStarted","Data":"ead85c989b9907d8f2fbbad17cfb0e5f4ca7a63e0d5e9aea453b72f3cd56e62c"}
May 29 03:04:16.127532 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:16.127495    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:16.144203 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:16.144152    2553 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc" podStartSLOduration=1.144138521 podStartE2EDuration="1.144138521s" podCreationTimestamp="2026-05-29 03:04:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 03:04:16.14363363 +0000 UTC m=+2197.007246159" watchObservedRunningTime="2026-05-29 03:04:16.144138521 +0000 UTC m=+2197.007751052"
May 29 03:04:16.283526 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:16.283497    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t8ct4_96b44fa7-1fa7-4811-8513-8d367a40ed40/node-ca/0.log"
May 29 03:04:17.112073 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:17.112042    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4bbx5b8_2120d8c1-9172-44cc-af86-c23c1ae2580e/istio-proxy/0.log"
May 29 03:04:17.314401 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:17.314376    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-s8rdr_3efd16c1-2a11-4eb6-a42f-e7326eea722a/istio-proxy/0.log"
May 29 03:04:17.956334 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:17.956308    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fzhb8_2a4ccab7-f230-42f0-86ba-97f21234b006/serve-healthcheck-canary/0.log"
May 29 03:04:18.435202 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:18.435171    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-htf5k_7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6/kube-rbac-proxy/0.log"
May 29 03:04:18.458427 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:18.458399    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-htf5k_7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6/exporter/0.log"
May 29 03:04:18.486435 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:18.486391    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-htf5k_7fd7dcc6-dbfa-4c24-a6d8-4557b88316b6/extractor/0.log"
May 29 03:04:20.671470 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:20.671436    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-55b468f665-9jlkr_deaaf730-c1d8-44c6-811a-e72ce67ed1c1/nginx/0.log"
May 29 03:04:20.871789 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:20.871753    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29667045-gxh6s_7bab2ee4-f3f9-4269-b3c9-12a8c6674aa3/cleanup/0.log"
May 29 03:04:20.907142 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:20.907106    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29667060-4nk6k_86ac4a71-d8bc-4e0b-b876-da3aaf66c845/cleanup/0.log"
May 29 03:04:21.281212 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:21.281184    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-784f8c8bfd-ggzqr_2974d6c0-41cb-4276-9890-a3b55af1f299/manager/0.log"
May 29 03:04:21.305306 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:21.305283    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-784f8c8bfd-ggzqr_2974d6c0-41cb-4276-9890-a3b55af1f299/copy-manifests/0.log"
May 29 03:04:22.141815 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:22.141785    2553 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-btflc/perf-node-gather-daemonset-pc8tc"
May 29 03:04:22.649500 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:22.649470    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-784859b99c-vm9hx_06b36631-9a0a-4467-a1f6-48bff9a4da97/manager/0.log"
May 29 03:04:29.199445 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:29.199414    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h6mc4_5ef02e9b-bceb-480a-93cc-dfaaa87e8f72/kube-multus-additional-cni-plugins/0.log"
May 29 03:04:29.221165 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:29.221129    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h6mc4_5ef02e9b-bceb-480a-93cc-dfaaa87e8f72/egress-router-binary-copy/0.log"
May 29 03:04:29.244706 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:29.244671    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h6mc4_5ef02e9b-bceb-480a-93cc-dfaaa87e8f72/cni-plugins/0.log"
May 29 03:04:29.267963 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:29.267940    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h6mc4_5ef02e9b-bceb-480a-93cc-dfaaa87e8f72/bond-cni-plugin/0.log"
May 29 03:04:29.292334 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:29.292303    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h6mc4_5ef02e9b-bceb-480a-93cc-dfaaa87e8f72/routeoverride-cni/0.log"
May 29 03:04:29.317589 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:29.317561    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h6mc4_5ef02e9b-bceb-480a-93cc-dfaaa87e8f72/whereabouts-cni-bincopy/0.log"
May 29 03:04:29.345207 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:29.345183    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h6mc4_5ef02e9b-bceb-480a-93cc-dfaaa87e8f72/whereabouts-cni/0.log"
May 29 03:04:29.753093 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:29.753067    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-t8gs7_f05a0fdc-25f4-4b4a-a1a1-362bc7aaf076/kube-multus/0.log"
May 29 03:04:29.779347 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:29.779322    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7sv8v_9a900d97-0943-4380-a1e5-85bd64a30ba3/network-metrics-daemon/0.log"
May 29 03:04:29.803645 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:29.803622    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7sv8v_9a900d97-0943-4380-a1e5-85bd64a30ba3/kube-rbac-proxy/0.log"
May 29 03:04:30.838935 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:30.838849    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2mq8b_b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6/ovn-controller/0.log"
May 29 03:04:30.868953 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:30.868923    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2mq8b_b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6/ovn-acl-logging/0.log"
May 29 03:04:30.890253 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:30.890217    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2mq8b_b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6/kube-rbac-proxy-node/0.log"
May 29 03:04:30.914635 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:30.914600    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2mq8b_b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6/kube-rbac-proxy-ovn-metrics/0.log"
May 29 03:04:30.935584 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:30.935557    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2mq8b_b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6/northd/0.log"
May 29 03:04:30.957052 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:30.957028    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2mq8b_b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6/nbdb/0.log"
May 29 03:04:30.982436 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:30.982410    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2mq8b_b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6/sbdb/0.log"
May 29 03:04:31.076582 ip-10-0-133-245 kubenswrapper[2553]: I0529 03:04:31.076554    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2mq8b_b44b7ac4-22d0-49f8-bde3-f8fbaa5264c6/ovnkube-controller/0.log"