Jul 09 02:32:38.609033 ip-10-0-129-236 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 09 02:32:38.609046 ip-10-0-129-236 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 09 02:32:38.609056 ip-10-0-129-236 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 09 02:32:38.609391 ip-10-0-129-236 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 09 02:32:48.845840 ip-10-0-129-236 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 09 02:32:48.845857 ip-10-0-129-236 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 5876428977824b668fb86c3872dcce2c --
Jul 09 02:34:45.502591 ip-10-0-129-236 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 02:34:46.012228 ip-10-0-129-236 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 02:34:46.012228 ip-10-0-129-236 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 02:34:46.012228 ip-10-0-129-236 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 02:34:46.012228 ip-10-0-129-236 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 02:34:46.012228 ip-10-0-129-236 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 02:34:46.012228 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.844040    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 02:34:46.012228 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851246    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 02:34:46.012228 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851265    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 02:34:46.012228 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851271    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 02:34:46.012228 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851277    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 02:34:46.012228 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851281    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 02:34:46.012228 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851285    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 02:34:46.012228 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851290    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 02:34:46.012228 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851294    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 02:34:45.916389 ip-10-0-129-236 systemd[1]: Started Kubernetes Kubelet.
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851298    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851303    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851307    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851311    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851330    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851334    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851338    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851342    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851352    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851356    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851361    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851366    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851373    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851378    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851382    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851386    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851390    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851394    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851399    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 02:34:46.014950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851403    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851415    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851419    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851423    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851427    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851431    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851434    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851438    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851442    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851446    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851450    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851454    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851458    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851462    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851466    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851469    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851474    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851478    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851482    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851485    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 02:34:46.031496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851490    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851494    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851498    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851502    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851508    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851513    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851518    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851522    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851526    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851530    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851534    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851538    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851542    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851547    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851551    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851555    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851561    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851567    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851571    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 02:34:46.032349 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851574    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851578    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851583    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851587    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851592    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851598    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851602    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851605    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851609    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851613    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851617    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851621    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851625    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851629    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851633    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851637    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851640    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851652    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851656    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851659    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 02:34:46.032958 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851663    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851667    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851672    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851675    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851679    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851684    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851688    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851692    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851695    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851700    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851704    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851707    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.851711    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852795    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852804    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852809    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852814    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852818    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852822    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852826    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 02:34:46.033944 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852830    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852834    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852838    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852842    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852845    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852849    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852853    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852857    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852860    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852864    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852876    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852880    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852889    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852893    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852897    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852901    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852905    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852909    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852913    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 02:34:46.034845 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852917    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852922    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852925    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852929    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852933    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852936    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852940    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852944    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852948    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852952    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852956    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852959    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852963    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852967    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852971    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852974    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852978    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852982    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852986    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 02:34:46.035727 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852989    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.852995    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853001    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853007    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853014    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853021    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853025    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853029    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853033    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853037    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853041    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853045    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853048    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853054    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853057    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853061    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853065    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853069    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853073    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 02:34:46.036614 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853077    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853080    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853084    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853088    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853092    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853095    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853099    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853103    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853106    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853110    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853115    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853119    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853123    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853127    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853131    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853135    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853139    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853142    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853146    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853151    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 02:34:46.037533 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853156    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853159    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853163    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853167    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853170    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853174    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853178    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853182    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853186    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853190    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853194    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853197    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853201    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853205    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853208    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853768    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853776    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853780    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853784    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853788    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 02:34:46.038421 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853792    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853796    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853800    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853803    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853808    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853812    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853816    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853820    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853824    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853828    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853832    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853836    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853840    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853844    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853849    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853852    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853856    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853860    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853864    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853868    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 02:34:46.039302 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853872    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853876    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853880    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853884    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853887    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853891    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853895    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853899    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853902    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853906    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853910    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853914    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853928    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853932    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853938    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853943    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853948    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853952    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853955    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 02:34:46.040204 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853961    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853965    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853969    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853975    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853980    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853985    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853993    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.853998    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854002    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854006    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854010    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854015    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854018    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854022    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854026    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854030    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854034    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854037    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854040    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 02:34:46.041024 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854047    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854052    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854055    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854061    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854065    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854068    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854072    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854075    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854079    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854083    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854086    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854090    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854093    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854097    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854101    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854104    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854108    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854112    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854116    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854120    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 02:34:46.041920 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854123    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854127    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854131    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854134    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854138    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854142    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854146    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854150    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854153    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854158    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854162    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854166    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854171    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854174    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854178    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.854183    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854275    2578 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854284    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854293    2578 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854300    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854305    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 02:34:46.042814 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854309    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854334    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854341    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854345    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854350    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854354    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854359    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854364    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854369    2578 flags.go:64] FLAG: --cgroup-root=""
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854373    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854377    2578 flags.go:64] FLAG: --client-ca-file=""
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854382    2578 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854387    2578 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854393    2578 flags.go:64] FLAG: --cluster-domain=""
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854397    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854401    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854406    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854411    2578 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854422    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854426    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854430    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854435    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854439    2578 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854445    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 02:34:46.043702 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854450    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854454    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854460    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854467    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854471    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854475    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854480    2578 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854484    2578 flags.go:64] FLAG: --enable-server="true"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854488    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854494    2578 flags.go:64] FLAG: --event-burst="100"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854499    2578 flags.go:64] FLAG: --event-qps="50"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854504    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854508    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854513    2578 flags.go:64] FLAG: --eviction-hard=""
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854519    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854522    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854527    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854532    2578 flags.go:64] FLAG: --eviction-soft=""
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854536    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854540    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854545    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854549    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854554    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854557    2578 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854562    2578 flags.go:64] FLAG: --feature-gates=""
Jul 09 02:34:46.044811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854567    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854572    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854576    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854580    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854585    2578 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854590    2578 flags.go:64] FLAG: --help="false"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854596    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854601    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854605    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854611    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854616    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854622    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854625    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854630    2578 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854634    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854638    2578 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854643    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854647    2578 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854652    2578 flags.go:64] FLAG: --kube-reserved=""
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854656    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854660    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854664    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854669    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854673    2578 flags.go:64] FLAG: --lock-file=""
Jul 09 02:34:46.045928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854677    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854681    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854685    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854693    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854698    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854703    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854707    2578 flags.go:64] FLAG: --logging-format="text"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854711    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854715    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854720    2578 flags.go:64] FLAG: --manifest-url=""
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854724    2578 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854730    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854734    2578 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854740    2578 flags.go:64] FLAG: --max-pods="110"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854745    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854750    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854754    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854760    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854765    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854769    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854773    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854783    2578 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854788    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854793    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 02:34:46.047031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854797    2578 flags.go:64] FLAG: --pod-cidr=""
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854801    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854809    2578 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854812    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854816    2578 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854821    2578 flags.go:64] FLAG: --port="10250"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854825    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854829    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cfd33011b1e489de"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854834    2578 flags.go:64] FLAG: --qos-reserved=""
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854838    2578 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854843    2578 flags.go:64] FLAG: --register-node="true"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854846    2578 flags.go:64] FLAG: --register-with-taints=""
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854851    2578 flags.go:64] FLAG: --registry-burst="10"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854856    2578 flags.go:64] FLAG: --registry-qps="5"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854861    2578 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854864    2578 flags.go:64] FLAG: --reserved-memory=""
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854870    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854875    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854879    2578 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854883    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854887    2578 flags.go:64] FLAG: --runonce="false"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854892    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854896    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854902    2578 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854907    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 02:34:46.051644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854912    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854916    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854921    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854925    2578 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854930    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854934    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854938    2578 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854942    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854947    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854952    2578 flags.go:64] FLAG: --system-cgroups=""
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854957    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854965    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854969    2578 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854973    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854978    2578 flags.go:64] FLAG: --tls-min-version=""
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854983    2578 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854986    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854991    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.854995    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.855000    2578 flags.go:64] FLAG: --v="2"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.855006    2578 flags.go:64] FLAG: --version="false"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.855012    2578 flags.go:64] FLAG: --vmodule=""
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.855018    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.855022    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855184    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 02:34:46.052764 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855189    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855193    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855198    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855202    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855206    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855212    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855216    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855223    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855228    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855233    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855238    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855242    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855246    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855250    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855255    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855258    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855262    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855267    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855270    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855274    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 02:34:46.060182 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855278    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855282    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855285    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855289    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855292    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855296    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855300    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855304    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855308    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855329    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855334    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855337    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855342    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855345    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855349    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855353    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855356    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855362    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855366    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855370    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 02:34:46.061082 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855374    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855378    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855382    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855385    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855389    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855393    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855397    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855401    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855404    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855408    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855412    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855416    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855420    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855424    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855428    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855432    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855436    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855439    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855443    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 02:34:46.061976 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855448    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855451    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855455    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855459    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855463    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855467    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855472    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855477    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855481    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855487    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855493    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855498    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855504    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855507    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855512    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855516    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855520    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855524    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855528    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 02:34:46.062872 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855533    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855537    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855540    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855544    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855548    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855552    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855555    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855559    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855563    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855567    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855571    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855575    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855579    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855582    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855586    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855590    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855593    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855598    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855602    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 02:34:46.063753 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.855605    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 02:34:46.151828 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.856339    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 02:34:46.151828 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.863629    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 09 02:34:46.151828 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.863643    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 02:34:46.151828 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.864425    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 02:34:46.151828 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.864437    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 02:34:46.151828 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864498    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 02:34:46.151828 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864505    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 02:34:46.151828 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864508    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 02:34:46.151828 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864511    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 02:34:46.151828 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864514    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 02:34:46.151828 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864517    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 02:34:46.151828 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864519    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 02:34:46.151828 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864522    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 02:34:46.151828 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864525    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864528    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864531    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864533    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864536    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864539    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864541    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864544    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864547    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864549    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864552    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864555    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864558    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864560    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864563    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864566    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864568    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864571    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864574    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864576    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 02:34:46.152617 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864579    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864582    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864584    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864587    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864590    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864592    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864594    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864597    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864599    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864602    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864604    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864607    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864609    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864612    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864615    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864617    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864620    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864622    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864625    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864628    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 02:34:46.153511 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864631    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864633    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864636    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864641    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864644    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864647    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864651    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864653    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864656    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864659    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864661    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864664    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864666    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864669    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864672    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864674    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864677    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864680    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864684    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 02:34:46.154620 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864687    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864689    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864692    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864694    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864697    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864699    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864701    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864704    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864707    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864709    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864713    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864715    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864718    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864721    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864723    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864726    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864728    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864731    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864734    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 02:34:46.155517 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864737    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 02:34:46.156608 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864739    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 02:34:46.156608 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864743    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 02:34:46.156608 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864747    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 02:34:46.156608 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864749    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 02:34:46.156608 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864753    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 02:34:46.156608 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864755    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 02:34:46.156608 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864758    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 02:34:46.156608 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864761    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 02:34:46.156608 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864764    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 02:34:46.156608 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864767    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 02:34:46.156608 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864770    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 02:34:46.156608 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864775    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 02:34:46.156608 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.864781    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 02:34:46.156608 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864926    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 02:34:46.156608 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864932    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864935    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864938    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864941    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864944    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864947    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864950    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864952    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864955    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864957    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864960    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864963    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864965    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864968    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864970    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864973    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864977    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864980    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864984    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 02:34:46.157694 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864986    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864989    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864992    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864995    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.864998    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865000    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865004    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865006    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865013    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865016    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865021    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865025    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865028    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865030    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865033    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865035    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865038    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865041    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865043    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865046    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 02:34:46.158590 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865048    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865051    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865053    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865056    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865058    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865061    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865063    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865066    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865070    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865073    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865076    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865079    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865082    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865085    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865087    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865090    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865093    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865095    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865098    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 02:34:46.159467 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865101    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865103    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865112    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865116    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865120    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865122    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865124    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865127    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865129    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865132    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865134    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865137    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865139    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865142    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865146    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865149    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865152    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865154    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865157    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865159    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 02:34:46.160567 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865162    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865164    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865166    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865169    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865171    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865174    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865176    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865179    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865181    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865184    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865186    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865189    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865191    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865194    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865196    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865201    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865205    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865207    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865210    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 02:34:46.161469 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:45.865212    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 02:34:46.162575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.865217    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 02:34:46.162575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.865327    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 02:34:46.162575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.867433    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 02:34:46.162575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.868744    2578 server.go:1019] "Starting client certificate rotation"
Jul 09 02:34:46.162575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.868843    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 02:34:46.162575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.868880    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 02:34:46.162575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.871117    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 02:34:46.162575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.872247    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 02:34:46.162575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.882337    2578 log.go:25] "Validated CRI v1 runtime API"
Jul 09 02:34:46.162575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.886284    2578 log.go:25] "Validated CRI v1 image API"
Jul 09 02:34:46.162575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.887137    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 02:34:46.162575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.889980    2578 fs.go:135] Filesystem UUIDs: map[22cdb391-303d-4fd8-9b79-547f89ad1fc3:/dev/nvme0n1p4 24f4565a-b647-4a39-9034-6694ed606f25:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 09 02:34:46.163241 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.889993    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 02:34:46.163669 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.895396    2578 manager.go:216] Machine: {Timestamp:2026-07-09 02:34:45.894240751 +0000 UTC m=+0.288044156 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104380 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c7ca03a25d0db47a651f17ccae276 SystemUUID:ec2c7ca0-3a25-d0db-47a6-51f17ccae276 BootID:58764289-7782-4b66-8fb8-6c3872dcce2c Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b2:26:3a:90:5b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b2:26:3a:90:5b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:29:84:fa:75:fa Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 02:34:46.163669 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.895497    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 02:34:46.163669 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.895593    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 02:34:46.163669 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.898132    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 02:34:46.163669 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.898162    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-236.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 02:34:46.163669 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.899082    2578 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 02:34:46.163669 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.899092    2578 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 02:34:46.163669 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.899107    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.899742    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.900353    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.901660    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.905564    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.905669    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.905696    2578 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.905705    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.905725    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.905737    2578 kubelet.go:398] "Adding apiserver pod source"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.905749    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.906131    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.906152    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.907983    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.909983    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.910011    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.911098    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.911110    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.911116    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.911122    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.911131    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.911137    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.911145    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.911152    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.911160    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.911165    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.911173    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.911179    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.911989    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.912004    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.914948    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-236.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:45.914957    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-236.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:45.914965    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.915152    2578 server.go:1268] "Started kubelet"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.915218    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.915297    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.915378    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.916684    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.918080    2578 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 02:34:46.164347 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:45.920497    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.922847    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.922868    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.926047    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.925945    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.926113    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:45.926344    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-236.ec2.internal\" not found"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.926914    2578 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.926926    2578 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.927204    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4h26m"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.928002    2578 factory.go:153] Registering CRI-O factory
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.928017    2578 factory.go:223] Registration of the crio container factory successfully
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:45.922950    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-236.ec2.internal.18c07e7340f92c01  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-236.ec2.internal,UID:ip-10-0-129-236.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-236.ec2.internal,},FirstTimestamp:2026-07-09 02:34:45.915126785 +0000 UTC m=+0.308930189,LastTimestamp:2026-07-09 02:34:45.915126785 +0000 UTC m=+0.308930189,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-236.ec2.internal,}"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.928411    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.928425    2578 factory.go:55] Registering systemd factory
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.928435    2578 factory.go:223] Registration of the systemd container factory successfully
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.928457    2578 factory.go:103] Registering Raw factory
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.928472    2578 manager.go:1196] Started watching for new ooms in manager
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.928846    2578 manager.go:349] Starting recovery of all containers
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.931484    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4h26m"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:45.935532    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:45.935620    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-236.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.940908    2578 manager.go:354] Recovery completed
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.944932    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.946813    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.946841    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.946851    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasSufficientPID"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.947335    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.947346    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.947357    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.947365    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.948358    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 02:34:46.168336 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.948385    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.948394    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasSufficientPID"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:45.948687    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-236.ec2.internal.18c07e7342dce5e1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-236.ec2.internal,UID:ip-10-0-129-236.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-236.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-236.ec2.internal,},FirstTimestamp:2026-07-09 02:34:45.946828257 +0000 UTC m=+0.340631661,LastTimestamp:2026-07-09 02:34:45.946828257 +0000 UTC m=+0.340631661,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-236.ec2.internal,}"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.949460    2578 policy_none.go:49] "None policy: Start"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.949474    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.949491    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.951531    2578 policy_none.go:47] "Start"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.997927    2578 manager.go:337] "Starting Device Plugin manager"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:45.998135    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.998152    2578 server.go:87] "Starting device plugin registration server"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.998421    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.998434    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.998529    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.998679    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:45.998689    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:45.999189    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:45.999241    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-236.ec2.internal\" not found"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.098914    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.102200    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.102235    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.102252    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasSufficientPID"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.102287    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.109394    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:46.109417    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-236.ec2.internal\": node \"ip-10-0-129-236.ec2.internal\" not found"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.110050    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.111144    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.111177    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.111206    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:46.111239    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.113291    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 02:34:46.169685 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:46.162210    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-236.ec2.internal\" not found"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.211283    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-236.ec2.internal"]
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.211410    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.213358    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.213392    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.213406    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasSufficientPID"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.215832    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.215936    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.215965    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.216698    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.216724    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.216736    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasSufficientPID"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.216743    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.216764    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.216773    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasSufficientPID"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.219878    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.219905    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.223297    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.223336    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.223347    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeHasSufficientPID"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.228823    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fae37a88080d71f4e174849eeb91a89e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal\" (UID: \"fae37a88080d71f4e174849eeb91a89e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.228846    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4d17de08da950bb8f59f5515256e0343-config\") pod \"kube-apiserver-proxy-ip-10-0-129-236.ec2.internal\" (UID: \"4d17de08da950bb8f59f5515256e0343\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.228863    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fae37a88080d71f4e174849eeb91a89e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal\" (UID: \"fae37a88080d71f4e174849eeb91a89e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:46.242939    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-236.ec2.internal\" not found" node="ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:46.248671    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-236.ec2.internal\" not found" node="ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.269934 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:46.263239    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-236.ec2.internal\" not found"
Jul 09 02:34:46.391123 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.328974    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fae37a88080d71f4e174849eeb91a89e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal\" (UID: \"fae37a88080d71f4e174849eeb91a89e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.391123 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.329003    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fae37a88080d71f4e174849eeb91a89e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal\" (UID: \"fae37a88080d71f4e174849eeb91a89e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.391123 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.329026    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4d17de08da950bb8f59f5515256e0343-config\") pod \"kube-apiserver-proxy-ip-10-0-129-236.ec2.internal\" (UID: \"4d17de08da950bb8f59f5515256e0343\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.391123 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.329088    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fae37a88080d71f4e174849eeb91a89e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal\" (UID: \"fae37a88080d71f4e174849eeb91a89e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.391123 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.329090    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fae37a88080d71f4e174849eeb91a89e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal\" (UID: \"fae37a88080d71f4e174849eeb91a89e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.391123 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.329151    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4d17de08da950bb8f59f5515256e0343-config\") pod \"kube-apiserver-proxy-ip-10-0-129-236.ec2.internal\" (UID: \"4d17de08da950bb8f59f5515256e0343\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.391123 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:46.364055    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-236.ec2.internal\" not found"
Jul 09 02:34:46.526432 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:46.464851    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-236.ec2.internal\" not found"
Jul 09 02:34:46.666460 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.546121    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.666460 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.551437    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-236.ec2.internal"
Jul 09 02:34:46.666460 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:46.565950    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-236.ec2.internal\" not found"
Jul 09 02:34:46.668145 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:46.667805    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-236.ec2.internal\" not found"
Jul 09 02:34:46.840364 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.727226    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 02:34:46.840364 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:46.768847    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-236.ec2.internal\" not found"
Jul 09 02:34:46.969204 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.868575    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 02:34:46.969204 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.868702    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 02:34:46.969204 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.868736    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 02:34:46.969204 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:46.869637    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-236.ec2.internal\" not found"
Jul 09 02:34:46.969204 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.936303    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 02:29:46 +0000 UTC" deadline="2028-02-19 06:14:06.712667143 +0000 UTC"
Jul 09 02:34:46.969204 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:46.936346    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14163h39m19.776324147s"
Jul 09 02:34:46.970663 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:46.970517    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-236.ec2.internal\" not found"
Jul 09 02:34:47.085905 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:47.071621    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-236.ec2.internal\" not found"
Jul 09 02:34:47.237158 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:47.171784    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-236.ec2.internal\" not found"
Jul 09 02:34:47.237158 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:47.180723    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d17de08da950bb8f59f5515256e0343.slice/crio-45faa4b5f1b313649b16a4945b331319592854f72d61c2006ed4d3ff2fa70fa8 WatchSource:0}: Error finding container 45faa4b5f1b313649b16a4945b331319592854f72d61c2006ed4d3ff2fa70fa8: Status 404 returned error can't find the container with id 45faa4b5f1b313649b16a4945b331319592854f72d61c2006ed4d3ff2fa70fa8
Jul 09 02:34:47.237158 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.186963    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 02:34:47.255914 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.254194    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 02:34:47.448566 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:47.272496    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-236.ec2.internal\" not found"
Jul 09 02:34:47.448566 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:47.373639    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-236.ec2.internal\" not found"
Jul 09 02:34:47.448566 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.381610    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 02:34:47.448566 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.426637    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal"
Jul 09 02:34:47.448566 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.440510    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 02:34:47.448566 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.441464    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-236.ec2.internal"
Jul 09 02:34:47.448566 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.447899    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 02:34:47.907522 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.907483    2578 apiserver.go:52] "Watching apiserver"
Jul 09 02:34:47.913676 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.913641    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 02:34:47.914162 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.914125    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-4rnpt","openshift-multus/multus-additional-cni-plugins-8rhkt","openshift-multus/network-metrics-daemon-jt6w4","openshift-network-operator/iptables-alerter-kpjk2","openshift-ovn-kubernetes/ovnkube-node-6gjwz","kube-system/global-pull-secret-syncer-t6lpr","kube-system/konnectivity-agent-pzqxx","kube-system/kube-apiserver-proxy-ip-10-0-129-236.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x","openshift-image-registry/node-ca-kxrl5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal","openshift-multus/multus-5jtrv","openshift-network-diagnostics/network-check-target-ppkl8","openshift-cluster-node-tuning-operator/tuned-9qdgf"]
Jul 09 02:34:47.917225 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.917205    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pzqxx"
Jul 09 02:34:47.921414 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.920494    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:34:47.921414 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:47.920568    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:34:47.921414 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.920900    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 02:34:47.921414 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.921099    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 02:34:47.921414 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.921263    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mdm85\""
Jul 09 02:34:47.924247 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.923679    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:34:47.924247 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:47.923748    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:34:47.924247 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.923788    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 02:34:47.925167 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.924987    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kpjk2"
Jul 09 02:34:47.930755 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.929731    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-d8brb\""
Jul 09 02:34:47.930755 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.930016    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 02:34:47.930755 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.930361    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 02:34:47.931277 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.931034    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 02:34:47.931592 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.931437    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.934129 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.934024    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 02:34:47.934129 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.934078    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 02:34:47.934391 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.934323    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:47.934484 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.934445    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 02:34:47.934843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.934551    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 02:34:47.934843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.934709    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z7zm5\""
Jul 09 02:34:47.934843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.934800    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 02:34:47.935147 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.934990    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 02:34:47.936539 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.936466    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-d67tq\""
Jul 09 02:34:47.937076 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.936701    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 02:34:47.938089 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.937230    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 02:34:47.938089 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.937967    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:47.939154 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.939132    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 02:34:47.939721 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.939695    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 02:34:47.940816 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.940074    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-66z7w\""
Jul 09 02:34:47.940816 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.940104    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 02:34:47.940816 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.940328    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 02:34:47.940816 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.940691    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b55deb49-73ce-4694-9271-21097d8294d1-iptables-alerter-script\") pod \"iptables-alerter-kpjk2\" (UID: \"b55deb49-73ce-4694-9271-21097d8294d1\") " pod="openshift-network-operator/iptables-alerter-kpjk2"
Jul 09 02:34:47.940816 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.940723    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-run-netns\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.940816 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.940751    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-run-openvswitch\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.940816 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.940802    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-log-socket\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.941254 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.940884    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4rnpt"
Jul 09 02:34:47.941518 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941415    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d33c8ed2-57be-4728-bf4e-52cc94d3e001-ovnkube-script-lib\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.941518 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941452    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1508f1fe-46da-4bfa-bbd0-a975c19cf63b-konnectivity-ca\") pod \"konnectivity-agent-pzqxx\" (UID: \"1508f1fe-46da-4bfa-bbd0-a975c19cf63b\") " pod="kube-system/konnectivity-agent-pzqxx"
Jul 09 02:34:47.941518 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941487    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-kubelet\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.941733 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941537    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-run-systemd\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.941733 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941584    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d33c8ed2-57be-4728-bf4e-52cc94d3e001-ovn-node-metrics-cert\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.941733 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941616    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-systemd-units\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.941733 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941642    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-run-ovn\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.941733 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941668    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz2s9\" (UniqueName: \"kubernetes.io/projected/d33c8ed2-57be-4728-bf4e-52cc94d3e001-kube-api-access-sz2s9\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.941733 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d33c8ed2-57be-4728-bf4e-52cc94d3e001-ovnkube-config\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.941733 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941718    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs\") pod \"network-metrics-daemon-jt6w4\" (UID: \"29ffb437-ae69-4765-85d3-5f1ebe2c0346\") " pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:34:47.942090 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941752    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhwq5\" (UniqueName: \"kubernetes.io/projected/b55deb49-73ce-4694-9271-21097d8294d1-kube-api-access-hhwq5\") pod \"iptables-alerter-kpjk2\" (UID: \"b55deb49-73ce-4694-9271-21097d8294d1\") " pod="openshift-network-operator/iptables-alerter-kpjk2"
Jul 09 02:34:47.942090 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941777    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-etc-openvswitch\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.942090 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941806    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dwnz\" (UniqueName: \"kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz\") pod \"network-check-target-ppkl8\" (UID: \"f6f3333d-e691-4ae1-8ee6-55f30813e35d\") " pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:34:47.942090 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941831    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmrp5\" (UniqueName: \"kubernetes.io/projected/29ffb437-ae69-4765-85d3-5f1ebe2c0346-kube-api-access-qmrp5\") pod \"network-metrics-daemon-jt6w4\" (UID: \"29ffb437-ae69-4765-85d3-5f1ebe2c0346\") " pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:34:47.942090 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941860    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-node-log\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.942090 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941900    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-cni-bin\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.942090 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941931    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d33c8ed2-57be-4728-bf4e-52cc94d3e001-env-overrides\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.942090 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941957    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1508f1fe-46da-4bfa-bbd0-a975c19cf63b-agent-certs\") pod \"konnectivity-agent-pzqxx\" (UID: \"1508f1fe-46da-4bfa-bbd0-a975c19cf63b\") " pod="kube-system/konnectivity-agent-pzqxx"
Jul 09 02:34:47.942090 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.941989    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b55deb49-73ce-4694-9271-21097d8294d1-host-slash\") pod \"iptables-alerter-kpjk2\" (UID: \"b55deb49-73ce-4694-9271-21097d8294d1\") " pod="openshift-network-operator/iptables-alerter-kpjk2"
Jul 09 02:34:47.942090 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.942012    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-var-lib-openvswitch\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.942090 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.942042    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-run-ovn-kubernetes\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.942090 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.942067    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-cni-netd\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.942090 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.942093    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.942804 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.942119    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-slash\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:47.944183 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.943143    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 02:34:47.944183 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.943368    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ftqjs\""
Jul 09 02:34:47.944183 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.943638    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 02:34:47.944183 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.943714    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:47.945502 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.945470    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kxrl5"
Jul 09 02:34:47.946419 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.946001    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fwrm6\""
Jul 09 02:34:47.946419 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.946211    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 02:34:47.946419 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.946251    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 02:34:47.946652 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.946523    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 02:34:47.946709 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.946689    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 02:34:47.946859 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.946835    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 02:34:47.947245 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.947223    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 02:34:47.947567 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.947547    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 02:34:47.947775 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.947740    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mcvwv\""
Jul 09 02:34:47.948066 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.947981    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 02:34:47.948665 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.948632    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:47.950661 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.950641    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c8t7j\""
Jul 09 02:34:47.950804 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.950782    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 02:34:47.951262 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.951185    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:47.951387 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:47.951257    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:34:47.958740 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.958716    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bqldj"
Jul 09 02:34:47.967649 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:47.967631    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bqldj"
Jul 09 02:34:48.020798 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.020598    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 02:34:48.027596 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.027576    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 02:34:48.043511 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043428    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-var-lib-cni-multus\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.043511 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043478    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1508f1fe-46da-4bfa-bbd0-a975c19cf63b-konnectivity-ca\") pod \"konnectivity-agent-pzqxx\" (UID: \"1508f1fe-46da-4bfa-bbd0-a975c19cf63b\") " pod="kube-system/konnectivity-agent-pzqxx"
Jul 09 02:34:48.043511 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043507    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-kubelet\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.043726 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043528    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d33c8ed2-57be-4728-bf4e-52cc94d3e001-ovn-node-metrics-cert\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.043726 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043545    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-sysctl-d\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.043726 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043569    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-kubelet\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.043726 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043658    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-run-ovn\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.043726 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043701    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz2s9\" (UniqueName: \"kubernetes.io/projected/d33c8ed2-57be-4728-bf4e-52cc94d3e001-kube-api-access-sz2s9\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.043726 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043713    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-run-ovn\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.043726 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043729    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-sys-fs\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.044063 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043757    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e59810c5-e353-4644-b26c-404664df8c06-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.044063 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043789    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-multus-socket-dir-parent\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.044063 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043815    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d33c8ed2-57be-4728-bf4e-52cc94d3e001-ovnkube-config\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.044063 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043841    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs\") pod \"network-metrics-daemon-jt6w4\" (UID: \"29ffb437-ae69-4765-85d3-5f1ebe2c0346\") " pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:34:48.044063 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043873    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-etc-openvswitch\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.044063 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043903    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-tuned\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.044063 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043929    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d99n9\" (UniqueName: \"kubernetes.io/projected/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-kube-api-access-d99n9\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.044063 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043931    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 02:34:48.044063 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043955    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dwnz\" (UniqueName: \"kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz\") pod \"network-check-target-ppkl8\" (UID: \"f6f3333d-e691-4ae1-8ee6-55f30813e35d\") " pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:34:48.044063 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043963    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-etc-openvswitch\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.044063 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.043978    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-cni-bin\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.044063 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044001    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/147eafc1-fac7-4201-b3d1-261371a5667a-host\") pod \"node-ca-kxrl5\" (UID: \"147eafc1-fac7-4201-b3d1-261371a5667a\") " pod="openshift-image-registry/node-ca-kxrl5"
Jul 09 02:34:48.044063 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044024    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b55deb49-73ce-4694-9271-21097d8294d1-host-slash\") pod \"iptables-alerter-kpjk2\" (UID: \"b55deb49-73ce-4694-9271-21097d8294d1\") " pod="openshift-network-operator/iptables-alerter-kpjk2"
Jul 09 02:34:48.044063 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044048    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-run-ovn-kubernetes\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.044063 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:48.044058    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 02:34:48.044835 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044071    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-cni-netd\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.044835 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044095    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/759cd68b-755a-47d5-bb83-7a9028adbc86-dbus\") pod \"global-pull-secret-syncer-t6lpr\" (UID: \"759cd68b-755a-47d5-bb83-7a9028adbc86\") " pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:48.044835 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044117    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret\") pod \"global-pull-secret-syncer-t6lpr\" (UID: \"759cd68b-755a-47d5-bb83-7a9028adbc86\") " pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:48.044835 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:48.044153    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs podName:29ffb437-ae69-4765-85d3-5f1ebe2c0346 nodeName:}" failed. No retries permitted until 2026-07-09 02:34:48.54411521 +0000 UTC m=+2.937918623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs") pod "network-metrics-daemon-jt6w4" (UID: "29ffb437-ae69-4765-85d3-5f1ebe2c0346") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 02:34:48.044835 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044202    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-tmp\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.044835 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044210    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1508f1fe-46da-4bfa-bbd0-a975c19cf63b-konnectivity-ca\") pod \"konnectivity-agent-pzqxx\" (UID: \"1508f1fe-46da-4bfa-bbd0-a975c19cf63b\") " pod="kube-system/konnectivity-agent-pzqxx"
Jul 09 02:34:48.044835 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044235    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-registration-dir\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.044835 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044263    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwbvq\" (UniqueName: \"kubernetes.io/projected/77446bdb-57d3-4db7-8518-63cdb3111bd9-kube-api-access-gwbvq\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.044835 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044288    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-device-dir\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.044835 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044315    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-os-release\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.044835 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044318    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b55deb49-73ce-4694-9271-21097d8294d1-host-slash\") pod \"iptables-alerter-kpjk2\" (UID: \"b55deb49-73ce-4694-9271-21097d8294d1\") " pod="openshift-network-operator/iptables-alerter-kpjk2"
Jul 09 02:34:48.044835 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044341    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-run-k8s-cni-cncf-io\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.044835 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044390    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6-hosts-file\") pod \"node-resolver-4rnpt\" (UID: \"daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6\") " pod="openshift-dns/node-resolver-4rnpt"
Jul 09 02:34:48.045521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044376    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-cni-bin\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.045521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044421    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-run-ovn-kubernetes\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.045521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044474    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-slash\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.045521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044489    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-cni-netd\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.045521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044530    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-run-netns\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.045521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044553    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-slash\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.045521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044560    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-multus-conf-dir\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.045521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044587    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d33c8ed2-57be-4728-bf4e-52cc94d3e001-ovnkube-config\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.045521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044598    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e59810c5-e353-4644-b26c-404664df8c06-cnibin\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.045521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044605    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-run-netns\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.045521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044631    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-etc-kubernetes\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.045521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044678    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-run-systemd\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.045521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044725    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/759cd68b-755a-47d5-bb83-7a9028adbc86-kubelet-config\") pod \"global-pull-secret-syncer-t6lpr\" (UID: \"759cd68b-755a-47d5-bb83-7a9028adbc86\") " pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:48.045521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044761    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-kubernetes\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.046193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-var-lib-kubelet\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.046193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044795    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-run-systemd\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.046193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044822    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.046193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044847    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-etc-selinux\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.046193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044893    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e59810c5-e353-4644-b26c-404664df8c06-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.046193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044925    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p6p2\" (UniqueName: \"kubernetes.io/projected/e59810c5-e353-4644-b26c-404664df8c06-kube-api-access-6p6p2\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.046193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044946    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-systemd-units\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.046193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.044966    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-sysconfig\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.046193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045005    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-systemd\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.046193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045002    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-systemd-units\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.046193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045035    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-run\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.046193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045073    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-sys\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.046193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045110    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e59810c5-e353-4644-b26c-404664df8c06-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.046193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045136    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-system-cni-dir\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.046946 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045161    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-multus-cni-dir\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.046946 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhwq5\" (UniqueName: \"kubernetes.io/projected/b55deb49-73ce-4694-9271-21097d8294d1-kube-api-access-hhwq5\") pod \"iptables-alerter-kpjk2\" (UID: \"b55deb49-73ce-4694-9271-21097d8294d1\") " pod="openshift-network-operator/iptables-alerter-kpjk2"
Jul 09 02:34:48.046946 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045225    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-var-lib-kubelet\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.046946 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045259    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-hostroot\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.046946 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045424    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-run-multus-certs\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.046946 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045467    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmrp5\" (UniqueName: \"kubernetes.io/projected/29ffb437-ae69-4765-85d3-5f1ebe2c0346-kube-api-access-qmrp5\") pod \"network-metrics-daemon-jt6w4\" (UID: \"29ffb437-ae69-4765-85d3-5f1ebe2c0346\") " pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:34:48.046946 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045506    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-node-log\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.046946 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045530    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d33c8ed2-57be-4728-bf4e-52cc94d3e001-env-overrides\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.046946 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045575    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-node-log\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.046946 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045576    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-lib-modules\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.046946 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045634    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e59810c5-e353-4644-b26c-404664df8c06-cni-binary-copy\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.046946 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045662    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-cnibin\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.046946 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045686    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77446bdb-57d3-4db7-8518-63cdb3111bd9-cni-binary-copy\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.046946 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045717    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77446bdb-57d3-4db7-8518-63cdb3111bd9-multus-daemon-config\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.047682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045753    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1508f1fe-46da-4bfa-bbd0-a975c19cf63b-agent-certs\") pod \"konnectivity-agent-pzqxx\" (UID: \"1508f1fe-46da-4bfa-bbd0-a975c19cf63b\") " pod="kube-system/konnectivity-agent-pzqxx"
Jul 09 02:34:48.047682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045787    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-var-lib-openvswitch\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.047682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045813    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-host\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.047682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045837    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-socket-dir\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.047682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045862    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2ht6\" (UniqueName: \"kubernetes.io/projected/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-kube-api-access-w2ht6\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.047682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045887    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6-tmp-dir\") pod \"node-resolver-4rnpt\" (UID: \"daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6\") " pod="openshift-dns/node-resolver-4rnpt"
Jul 09 02:34:48.047682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045891    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-var-lib-openvswitch\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.047682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045916    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqll7\" (UniqueName: \"kubernetes.io/projected/daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6-kube-api-access-gqll7\") pod \"node-resolver-4rnpt\" (UID: \"daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6\") " pod="openshift-dns/node-resolver-4rnpt"
Jul 09 02:34:48.047682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045945    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6xzs\" (UniqueName: \"kubernetes.io/projected/147eafc1-fac7-4201-b3d1-261371a5667a-kube-api-access-p6xzs\") pod \"node-ca-kxrl5\" (UID: \"147eafc1-fac7-4201-b3d1-261371a5667a\") " pod="openshift-image-registry/node-ca-kxrl5"
Jul 09 02:34:48.047682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045974    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d33c8ed2-57be-4728-bf4e-52cc94d3e001-env-overrides\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.047682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.045996    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.047682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.046045    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.047682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.046044    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-modprobe-d\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.047682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.046082    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/147eafc1-fac7-4201-b3d1-261371a5667a-serviceca\") pod \"node-ca-kxrl5\" (UID: \"147eafc1-fac7-4201-b3d1-261371a5667a\") " pod="openshift-image-registry/node-ca-kxrl5"
Jul 09 02:34:48.048444 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.046116    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b55deb49-73ce-4694-9271-21097d8294d1-iptables-alerter-script\") pod \"iptables-alerter-kpjk2\" (UID: \"b55deb49-73ce-4694-9271-21097d8294d1\") " pod="openshift-network-operator/iptables-alerter-kpjk2"
Jul 09 02:34:48.048444 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.046151    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-run-openvswitch\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.048444 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.046177    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-run-netns\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.048444 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.046202    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-var-lib-cni-bin\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.048444 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.046260    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-run-openvswitch\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.048444 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.046329    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-log-socket\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.048444 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.046365    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d33c8ed2-57be-4728-bf4e-52cc94d3e001-ovnkube-script-lib\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.048444 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.046396    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-sysctl-conf\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.048444 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.046366    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d33c8ed2-57be-4728-bf4e-52cc94d3e001-log-socket\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.048444 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.046425    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e59810c5-e353-4644-b26c-404664df8c06-system-cni-dir\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.048444 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.046452    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e59810c5-e353-4644-b26c-404664df8c06-os-release\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.048444 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.046680    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b55deb49-73ce-4694-9271-21097d8294d1-iptables-alerter-script\") pod \"iptables-alerter-kpjk2\" (UID: \"b55deb49-73ce-4694-9271-21097d8294d1\") " pod="openshift-network-operator/iptables-alerter-kpjk2"
Jul 09 02:34:48.048444 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.046836    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d33c8ed2-57be-4728-bf4e-52cc94d3e001-ovnkube-script-lib\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.048444 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.048414    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d33c8ed2-57be-4728-bf4e-52cc94d3e001-ovn-node-metrics-cert\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.049446 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.049427    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1508f1fe-46da-4bfa-bbd0-a975c19cf63b-agent-certs\") pod \"konnectivity-agent-pzqxx\" (UID: \"1508f1fe-46da-4bfa-bbd0-a975c19cf63b\") " pod="kube-system/konnectivity-agent-pzqxx"
Jul 09 02:34:48.054362 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:48.054321    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 02:34:48.054362 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:48.054366    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 02:34:48.054567 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:48.054387    2578 projected.go:196] Error preparing data for projected volume kube-api-access-7dwnz for pod openshift-network-diagnostics/network-check-target-ppkl8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 02:34:48.054567 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:48.054456    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz podName:f6f3333d-e691-4ae1-8ee6-55f30813e35d nodeName:}" failed. No retries permitted until 2026-07-09 02:34:48.554438255 +0000 UTC m=+2.948241655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7dwnz" (UniqueName: "kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz") pod "network-check-target-ppkl8" (UID: "f6f3333d-e691-4ae1-8ee6-55f30813e35d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 02:34:48.055605 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.055552    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz2s9\" (UniqueName: \"kubernetes.io/projected/d33c8ed2-57be-4728-bf4e-52cc94d3e001-kube-api-access-sz2s9\") pod \"ovnkube-node-6gjwz\" (UID: \"d33c8ed2-57be-4728-bf4e-52cc94d3e001\") " pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.058076 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.058032    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhwq5\" (UniqueName: \"kubernetes.io/projected/b55deb49-73ce-4694-9271-21097d8294d1-kube-api-access-hhwq5\") pod \"iptables-alerter-kpjk2\" (UID: \"b55deb49-73ce-4694-9271-21097d8294d1\") " pod="openshift-network-operator/iptables-alerter-kpjk2"
Jul 09 02:34:48.058476 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.058431    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmrp5\" (UniqueName: \"kubernetes.io/projected/29ffb437-ae69-4765-85d3-5f1ebe2c0346-kube-api-access-qmrp5\") pod \"network-metrics-daemon-jt6w4\" (UID: \"29ffb437-ae69-4765-85d3-5f1ebe2c0346\") " pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:34:48.118984 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.118934    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal" event={"ID":"fae37a88080d71f4e174849eeb91a89e","Type":"ContainerStarted","Data":"b8b58885fe6e45cbfa9730086bc69a8201044d37610895964b7ae35b85cc1cd1"}
Jul 09 02:34:48.121003 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.120975    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-236.ec2.internal" event={"ID":"4d17de08da950bb8f59f5515256e0343","Type":"ContainerStarted","Data":"45faa4b5f1b313649b16a4945b331319592854f72d61c2006ed4d3ff2fa70fa8"}
Jul 09 02:34:48.147462 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147435    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/759cd68b-755a-47d5-bb83-7a9028adbc86-kubelet-config\") pod \"global-pull-secret-syncer-t6lpr\" (UID: \"759cd68b-755a-47d5-bb83-7a9028adbc86\") " pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:48.147593 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147476    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-kubernetes\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.147593 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147501    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-var-lib-kubelet\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.147593 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147527    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.147593 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147553    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-etc-selinux\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.147593 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147574    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/759cd68b-755a-47d5-bb83-7a9028adbc86-kubelet-config\") pod \"global-pull-secret-syncer-t6lpr\" (UID: \"759cd68b-755a-47d5-bb83-7a9028adbc86\") " pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:48.147864 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147579    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e59810c5-e353-4644-b26c-404664df8c06-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.147864 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147597    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-kubernetes\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.147864 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147630    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-var-lib-kubelet\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.147864 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147644    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p6p2\" (UniqueName: \"kubernetes.io/projected/e59810c5-e353-4644-b26c-404664df8c06-kube-api-access-6p6p2\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.147864 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147663    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.147864 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147684    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-sysconfig\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.147864 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147712    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-systemd\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.147864 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147719    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-etc-selinux\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.147864 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147738    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-run\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.147864 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-sys\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.147864 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147771    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-sysconfig\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.147864 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147774    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-systemd\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.147864 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147803    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e59810c5-e353-4644-b26c-404664df8c06-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.148560 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147882    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-run\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.148560 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147915    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-system-cni-dir\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.148560 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147936    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-sys\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.148560 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147965    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-system-cni-dir\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.148560 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.147976    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-multus-cni-dir\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.148560 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-var-lib-kubelet\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.148560 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148032    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-hostroot\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.148560 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148056    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-run-multus-certs\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.148560 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148090    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-lib-modules\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.148560 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148105    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-var-lib-kubelet\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.148560 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148117    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e59810c5-e353-4644-b26c-404664df8c06-cni-binary-copy\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.148560 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148122    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-multus-cni-dir\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.148560 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148168    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-hostroot\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.148560 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148172    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-cnibin\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.148560 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77446bdb-57d3-4db7-8518-63cdb3111bd9-cni-binary-copy\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.149023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148204    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-run-multus-certs\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.149023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148225    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77446bdb-57d3-4db7-8518-63cdb3111bd9-multus-daemon-config\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.149023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148241    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-lib-modules\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.149023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148262    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-cnibin\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.149023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148376    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e59810c5-e353-4644-b26c-404664df8c06-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.149023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148397    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-host\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.149023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148433    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-socket-dir\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.149023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148455    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e59810c5-e353-4644-b26c-404664df8c06-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.149023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148465    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2ht6\" (UniqueName: \"kubernetes.io/projected/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-kube-api-access-w2ht6\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.149023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148500    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6-tmp-dir\") pod \"node-resolver-4rnpt\" (UID: \"daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6\") " pod="openshift-dns/node-resolver-4rnpt"
Jul 09 02:34:48.149023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148551    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqll7\" (UniqueName: \"kubernetes.io/projected/daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6-kube-api-access-gqll7\") pod \"node-resolver-4rnpt\" (UID: \"daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6\") " pod="openshift-dns/node-resolver-4rnpt"
Jul 09 02:34:48.149023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148573    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6xzs\" (UniqueName: \"kubernetes.io/projected/147eafc1-fac7-4201-b3d1-261371a5667a-kube-api-access-p6xzs\") pod \"node-ca-kxrl5\" (UID: \"147eafc1-fac7-4201-b3d1-261371a5667a\") " pod="openshift-image-registry/node-ca-kxrl5"
Jul 09 02:34:48.149023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148590    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-modprobe-d\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.149023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148608    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/147eafc1-fac7-4201-b3d1-261371a5667a-serviceca\") pod \"node-ca-kxrl5\" (UID: \"147eafc1-fac7-4201-b3d1-261371a5667a\") " pod="openshift-image-registry/node-ca-kxrl5"
Jul 09 02:34:48.149718 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148627    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-run-netns\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.149718 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148678    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-var-lib-cni-bin\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.149718 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148710    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-sysctl-conf\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.149718 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148740    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e59810c5-e353-4644-b26c-404664df8c06-system-cni-dir\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.149718 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148756    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e59810c5-e353-4644-b26c-404664df8c06-os-release\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.149718 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148786    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-host\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.149718 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148829    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77446bdb-57d3-4db7-8518-63cdb3111bd9-multus-daemon-config\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.149718 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148875    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-var-lib-cni-multus\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.149718 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148909    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e59810c5-e353-4644-b26c-404664df8c06-cni-binary-copy\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.149718 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148913    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-run-netns\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.149718 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148914    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-sysctl-d\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.149718 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.148997    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-sys-fs\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.149718 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149012    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-modprobe-d\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.149718 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149024    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-sysctl-d\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.149718 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149046    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e59810c5-e353-4644-b26c-404664df8c06-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.150537 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149073    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-multus-socket-dir-parent\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.150537 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149065    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-var-lib-cni-bin\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.150537 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149131    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-tuned\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.150537 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149161    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d99n9\" (UniqueName: \"kubernetes.io/projected/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-kube-api-access-d99n9\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.150537 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149152    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-multus-socket-dir-parent\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.150537 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149241    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e59810c5-e353-4644-b26c-404664df8c06-os-release\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.150537 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149261    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-sys-fs\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.150537 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149260    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/147eafc1-fac7-4201-b3d1-261371a5667a-host\") pod \"node-ca-kxrl5\" (UID: \"147eafc1-fac7-4201-b3d1-261371a5667a\") " pod="openshift-image-registry/node-ca-kxrl5"
Jul 09 02:34:48.150537 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149310    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e59810c5-e353-4644-b26c-404664df8c06-system-cni-dir\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.150537 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149328    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/759cd68b-755a-47d5-bb83-7a9028adbc86-dbus\") pod \"global-pull-secret-syncer-t6lpr\" (UID: \"759cd68b-755a-47d5-bb83-7a9028adbc86\") " pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:48.150537 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149380    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6-tmp-dir\") pod \"node-resolver-4rnpt\" (UID: \"daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6\") " pod="openshift-dns/node-resolver-4rnpt"
Jul 09 02:34:48.150537 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149381    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret\") pod \"global-pull-secret-syncer-t6lpr\" (UID: \"759cd68b-755a-47d5-bb83-7a9028adbc86\") " pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:48.150537 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149481    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-tmp\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.150537 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149484    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/147eafc1-fac7-4201-b3d1-261371a5667a-serviceca\") pod \"node-ca-kxrl5\" (UID: \"147eafc1-fac7-4201-b3d1-261371a5667a\") " pod="openshift-image-registry/node-ca-kxrl5"
Jul 09 02:34:48.150537 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:48.149496    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 02:34:48.151281 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149515    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-registration-dir\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.151281 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149546    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwbvq\" (UniqueName: \"kubernetes.io/projected/77446bdb-57d3-4db7-8518-63cdb3111bd9-kube-api-access-gwbvq\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.151281 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:48.149551    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret podName:759cd68b-755a-47d5-bb83-7a9028adbc86 nodeName:}" failed. No retries permitted until 2026-07-09 02:34:48.64953923 +0000 UTC m=+3.043342641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret") pod "global-pull-secret-syncer-t6lpr" (UID: "759cd68b-755a-47d5-bb83-7a9028adbc86") : object "kube-system"/"original-pull-secret" not registered
Jul 09 02:34:48.151281 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149584    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-device-dir\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.151281 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149383    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-sysctl-conf\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.151281 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149587    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-var-lib-cni-multus\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.151281 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149547    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-socket-dir\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.151281 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149628    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-os-release\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.151281 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149661    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-run-k8s-cni-cncf-io\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.151281 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149669    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-device-dir\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.151281 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149686    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-registration-dir\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.151281 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149498    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/759cd68b-755a-47d5-bb83-7a9028adbc86-dbus\") pod \"global-pull-secret-syncer-t6lpr\" (UID: \"759cd68b-755a-47d5-bb83-7a9028adbc86\") " pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:48.151281 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149694    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6-hosts-file\") pod \"node-resolver-4rnpt\" (UID: \"daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6\") " pod="openshift-dns/node-resolver-4rnpt"
Jul 09 02:34:48.151281 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149744    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-host-run-k8s-cni-cncf-io\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.152089 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149763    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-multus-conf-dir\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.152089 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149770    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-os-release\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.152089 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149792    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e59810c5-e353-4644-b26c-404664df8c06-cnibin\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.152089 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149816    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-etc-kubernetes\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.152089 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149824    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-multus-conf-dir\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.152089 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149853    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e59810c5-e353-4644-b26c-404664df8c06-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.152089 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149873    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/147eafc1-fac7-4201-b3d1-261371a5667a-host\") pod \"node-ca-kxrl5\" (UID: \"147eafc1-fac7-4201-b3d1-261371a5667a\") " pod="openshift-image-registry/node-ca-kxrl5"
Jul 09 02:34:48.152089 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149931    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e59810c5-e353-4644-b26c-404664df8c06-cnibin\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.152089 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.149970    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77446bdb-57d3-4db7-8518-63cdb3111bd9-etc-kubernetes\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.152089 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.150029    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6-hosts-file\") pod \"node-resolver-4rnpt\" (UID: \"daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6\") " pod="openshift-dns/node-resolver-4rnpt"
Jul 09 02:34:48.152089 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.150282    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77446bdb-57d3-4db7-8518-63cdb3111bd9-cni-binary-copy\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.152089 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.151992    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-tmp\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.152665 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.152535    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-etc-tuned\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.156973 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.156920    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqll7\" (UniqueName: \"kubernetes.io/projected/daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6-kube-api-access-gqll7\") pod \"node-resolver-4rnpt\" (UID: \"daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6\") " pod="openshift-dns/node-resolver-4rnpt"
Jul 09 02:34:48.157240 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.157215    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p6p2\" (UniqueName: \"kubernetes.io/projected/e59810c5-e353-4644-b26c-404664df8c06-kube-api-access-6p6p2\") pod \"multus-additional-cni-plugins-8rhkt\" (UID: \"e59810c5-e353-4644-b26c-404664df8c06\") " pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.157437 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.157385    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6xzs\" (UniqueName: \"kubernetes.io/projected/147eafc1-fac7-4201-b3d1-261371a5667a-kube-api-access-p6xzs\") pod \"node-ca-kxrl5\" (UID: \"147eafc1-fac7-4201-b3d1-261371a5667a\") " pod="openshift-image-registry/node-ca-kxrl5"
Jul 09 02:34:48.157935 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.157886    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2ht6\" (UniqueName: \"kubernetes.io/projected/3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9-kube-api-access-w2ht6\") pod \"aws-ebs-csi-driver-node-2bc7x\" (UID: \"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.159567 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.159548    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwbvq\" (UniqueName: \"kubernetes.io/projected/77446bdb-57d3-4db7-8518-63cdb3111bd9-kube-api-access-gwbvq\") pod \"multus-5jtrv\" (UID: \"77446bdb-57d3-4db7-8518-63cdb3111bd9\") " pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.159879 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.159858    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d99n9\" (UniqueName: \"kubernetes.io/projected/a8a39f98-fa17-46a6-b79a-c1d5e1304a39-kube-api-access-d99n9\") pod \"tuned-9qdgf\" (UID: \"a8a39f98-fa17-46a6-b79a-c1d5e1304a39\") " pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.235243 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.235215    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pzqxx"
Jul 09 02:34:48.246129 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:48.246103    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1508f1fe_46da_4bfa_bbd0_a975c19cf63b.slice/crio-8d875edb03a479b8ed60b44247f73dd7b645ee7acccff4ee079e85c46a9beb6e WatchSource:0}: Error finding container 8d875edb03a479b8ed60b44247f73dd7b645ee7acccff4ee079e85c46a9beb6e: Status 404 returned error can't find the container with id 8d875edb03a479b8ed60b44247f73dd7b645ee7acccff4ee079e85c46a9beb6e
Jul 09 02:34:48.249598 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.249581    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kpjk2"
Jul 09 02:34:48.260029 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.259999    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:34:48.261520 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:48.261500    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb55deb49_73ce_4694_9271_21097d8294d1.slice/crio-2185ba33750ee5fff81028b803c0715bb312d6c4e7a86090c4068ba99087f56b WatchSource:0}: Error finding container 2185ba33750ee5fff81028b803c0715bb312d6c4e7a86090c4068ba99087f56b: Status 404 returned error can't find the container with id 2185ba33750ee5fff81028b803c0715bb312d6c4e7a86090c4068ba99087f56b
Jul 09 02:34:48.267012 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.266991    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9qdgf"
Jul 09 02:34:48.270098 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:48.270076    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd33c8ed2_57be_4728_bf4e_52cc94d3e001.slice/crio-fbd358f3ef8586afcf9b9231b3f7db7b3f57ad334d1ddfcc2ab994e22f7cf153 WatchSource:0}: Error finding container fbd358f3ef8586afcf9b9231b3f7db7b3f57ad334d1ddfcc2ab994e22f7cf153: Status 404 returned error can't find the container with id fbd358f3ef8586afcf9b9231b3f7db7b3f57ad334d1ddfcc2ab994e22f7cf153
Jul 09 02:34:48.276690 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:48.276662    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8a39f98_fa17_46a6_b79a_c1d5e1304a39.slice/crio-d6727232f218c2f9de9ee193faa5751df49c02605d2d842e5ecdfa7707d73caf WatchSource:0}: Error finding container d6727232f218c2f9de9ee193faa5751df49c02605d2d842e5ecdfa7707d73caf: Status 404 returned error can't find the container with id d6727232f218c2f9de9ee193faa5751df49c02605d2d842e5ecdfa7707d73caf
Jul 09 02:34:48.279042 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.279017    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x"
Jul 09 02:34:48.289024 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.289000    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4rnpt"
Jul 09 02:34:48.290688 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:48.290663    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3935cfd9_d4ef_4a7d_8f7d_36b78e7b0cd9.slice/crio-8f08a14a64c315006f9b2cb185d9c200012f66755eb98ca7e53427a97a42190e WatchSource:0}: Error finding container 8f08a14a64c315006f9b2cb185d9c200012f66755eb98ca7e53427a97a42190e: Status 404 returned error can't find the container with id 8f08a14a64c315006f9b2cb185d9c200012f66755eb98ca7e53427a97a42190e
Jul 09 02:34:48.297765 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.297745    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8rhkt"
Jul 09 02:34:48.299836 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:48.299810    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaefa1ce_31aa_4af4_8ec5_6a5b969a2dc6.slice/crio-4df1f25fb262102ac6188a5eafc92a68c492d8aadc9eabe3adb93dac305061c9 WatchSource:0}: Error finding container 4df1f25fb262102ac6188a5eafc92a68c492d8aadc9eabe3adb93dac305061c9: Status 404 returned error can't find the container with id 4df1f25fb262102ac6188a5eafc92a68c492d8aadc9eabe3adb93dac305061c9
Jul 09 02:34:48.308461 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.308364    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kxrl5"
Jul 09 02:34:48.309728 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:48.309697    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode59810c5_e353_4644_b26c_404664df8c06.slice/crio-7085567c68b250ddc01ca895cb0515b2e192b8ebfae59a8f12b7a9a784033f7c WatchSource:0}: Error finding container 7085567c68b250ddc01ca895cb0515b2e192b8ebfae59a8f12b7a9a784033f7c: Status 404 returned error can't find the container with id 7085567c68b250ddc01ca895cb0515b2e192b8ebfae59a8f12b7a9a784033f7c
Jul 09 02:34:48.316909 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.316887    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5jtrv"
Jul 09 02:34:48.319644 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:48.319612    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod147eafc1_fac7_4201_b3d1_261371a5667a.slice/crio-95beb6479abcf3797a5b1dc3210ab86fe3c8294c528cb872869458b3a17dff81 WatchSource:0}: Error finding container 95beb6479abcf3797a5b1dc3210ab86fe3c8294c528cb872869458b3a17dff81: Status 404 returned error can't find the container with id 95beb6479abcf3797a5b1dc3210ab86fe3c8294c528cb872869458b3a17dff81
Jul 09 02:34:48.326950 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:34:48.326930    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77446bdb_57d3_4db7_8518_63cdb3111bd9.slice/crio-f23a7f4482f26e92c692746e16737a5e1a3862cea2d50979023f23d2936d1b10 WatchSource:0}: Error finding container f23a7f4482f26e92c692746e16737a5e1a3862cea2d50979023f23d2936d1b10: Status 404 returned error can't find the container with id f23a7f4482f26e92c692746e16737a5e1a3862cea2d50979023f23d2936d1b10
Jul 09 02:34:48.553630 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.553548    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs\") pod \"network-metrics-daemon-jt6w4\" (UID: \"29ffb437-ae69-4765-85d3-5f1ebe2c0346\") " pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:34:48.553788 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:48.553741    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 02:34:48.553849 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:48.553825    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs podName:29ffb437-ae69-4765-85d3-5f1ebe2c0346 nodeName:}" failed. No retries permitted until 2026-07-09 02:34:49.5538043 +0000 UTC m=+3.947607694 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs") pod "network-metrics-daemon-jt6w4" (UID: "29ffb437-ae69-4765-85d3-5f1ebe2c0346") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 02:34:48.654894 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.654860    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dwnz\" (UniqueName: \"kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz\") pod \"network-check-target-ppkl8\" (UID: \"f6f3333d-e691-4ae1-8ee6-55f30813e35d\") " pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:34:48.655075 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.654902    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret\") pod \"global-pull-secret-syncer-t6lpr\" (UID: \"759cd68b-755a-47d5-bb83-7a9028adbc86\") " pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:48.655147 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:48.655065    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 02:34:48.655147 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:48.655094    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 02:34:48.655147 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:48.655109    2578 projected.go:196] Error preparing data for projected volume kube-api-access-7dwnz for pod openshift-network-diagnostics/network-check-target-ppkl8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 02:34:48.655147 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:48.655117    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 02:34:48.655333 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:48.655174    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz podName:f6f3333d-e691-4ae1-8ee6-55f30813e35d nodeName:}" failed. No retries permitted until 2026-07-09 02:34:49.655154869 +0000 UTC m=+4.048958266 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7dwnz" (UniqueName: "kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz") pod "network-check-target-ppkl8" (UID: "f6f3333d-e691-4ae1-8ee6-55f30813e35d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 02:34:48.655333 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:48.655196    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret podName:759cd68b-755a-47d5-bb83-7a9028adbc86 nodeName:}" failed. No retries permitted until 2026-07-09 02:34:49.65518751 +0000 UTC m=+4.048990907 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret") pod "global-pull-secret-syncer-t6lpr" (UID: "759cd68b-755a-47d5-bb83-7a9028adbc86") : object "kube-system"/"original-pull-secret" not registered
Jul 09 02:34:48.971463 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.971270    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 02:29:48 +0000 UTC" deadline="2028-04-22 17:13:25.62042163 +0000 UTC"
Jul 09 02:34:48.971463 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:48.971354    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15686h38m36.649072426s"
Jul 09 02:34:49.112081 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:49.112050    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:34:49.112341 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:49.112164    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:34:49.124305 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:49.124242    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kxrl5" event={"ID":"147eafc1-fac7-4201-b3d1-261371a5667a","Type":"ContainerStarted","Data":"95beb6479abcf3797a5b1dc3210ab86fe3c8294c528cb872869458b3a17dff81"}
Jul 09 02:34:49.127038 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:49.126944    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8rhkt" event={"ID":"e59810c5-e353-4644-b26c-404664df8c06","Type":"ContainerStarted","Data":"7085567c68b250ddc01ca895cb0515b2e192b8ebfae59a8f12b7a9a784033f7c"}
Jul 09 02:34:49.128694 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:49.128660    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz" event={"ID":"d33c8ed2-57be-4728-bf4e-52cc94d3e001","Type":"ContainerStarted","Data":"fbd358f3ef8586afcf9b9231b3f7db7b3f57ad334d1ddfcc2ab994e22f7cf153"}
Jul 09 02:34:49.130546 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:49.130459    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kpjk2" event={"ID":"b55deb49-73ce-4694-9271-21097d8294d1","Type":"ContainerStarted","Data":"2185ba33750ee5fff81028b803c0715bb312d6c4e7a86090c4068ba99087f56b"}
Jul 09 02:34:49.132430 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:49.132378    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5jtrv" event={"ID":"77446bdb-57d3-4db7-8518-63cdb3111bd9","Type":"ContainerStarted","Data":"f23a7f4482f26e92c692746e16737a5e1a3862cea2d50979023f23d2936d1b10"}
Jul 09 02:34:49.134212 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:49.134147    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4rnpt" event={"ID":"daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6","Type":"ContainerStarted","Data":"4df1f25fb262102ac6188a5eafc92a68c492d8aadc9eabe3adb93dac305061c9"}
Jul 09 02:34:49.135849 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:49.135786    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x" event={"ID":"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9","Type":"ContainerStarted","Data":"8f08a14a64c315006f9b2cb185d9c200012f66755eb98ca7e53427a97a42190e"}
Jul 09 02:34:49.137412 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:49.137385    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9qdgf" event={"ID":"a8a39f98-fa17-46a6-b79a-c1d5e1304a39","Type":"ContainerStarted","Data":"d6727232f218c2f9de9ee193faa5751df49c02605d2d842e5ecdfa7707d73caf"}
Jul 09 02:34:49.138878 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:49.138852    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pzqxx" event={"ID":"1508f1fe-46da-4bfa-bbd0-a975c19cf63b","Type":"ContainerStarted","Data":"8d875edb03a479b8ed60b44247f73dd7b645ee7acccff4ee079e85c46a9beb6e"}
Jul 09 02:34:49.561943 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:49.561901    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs\") pod \"network-metrics-daemon-jt6w4\" (UID: \"29ffb437-ae69-4765-85d3-5f1ebe2c0346\") " pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:34:49.562153 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:49.562104    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 02:34:49.562214 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:49.562175    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs podName:29ffb437-ae69-4765-85d3-5f1ebe2c0346 nodeName:}" failed. No retries permitted until 2026-07-09 02:34:51.562153785 +0000 UTC m=+5.955957183 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs") pod "network-metrics-daemon-jt6w4" (UID: "29ffb437-ae69-4765-85d3-5f1ebe2c0346") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 02:34:49.663317 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:49.663277    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret\") pod \"global-pull-secret-syncer-t6lpr\" (UID: \"759cd68b-755a-47d5-bb83-7a9028adbc86\") " pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:49.663507 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:49.663392    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dwnz\" (UniqueName: \"kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz\") pod \"network-check-target-ppkl8\" (UID: \"f6f3333d-e691-4ae1-8ee6-55f30813e35d\") " pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:34:49.663589 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:49.663536    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 02:34:49.663589 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:49.663560    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 02:34:49.663589 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:49.663575    2578 projected.go:196] Error preparing data for projected volume kube-api-access-7dwnz for pod openshift-network-diagnostics/network-check-target-ppkl8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 02:34:49.663732 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:49.663642    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz podName:f6f3333d-e691-4ae1-8ee6-55f30813e35d nodeName:}" failed. No retries permitted until 2026-07-09 02:34:51.663621395 +0000 UTC m=+6.057424791 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7dwnz" (UniqueName: "kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz") pod "network-check-target-ppkl8" (UID: "f6f3333d-e691-4ae1-8ee6-55f30813e35d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 02:34:49.664085 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:49.664058    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 02:34:49.664208 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:49.664120    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret podName:759cd68b-755a-47d5-bb83-7a9028adbc86 nodeName:}" failed. No retries permitted until 2026-07-09 02:34:51.66410361 +0000 UTC m=+6.057907019 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret") pod "global-pull-secret-syncer-t6lpr" (UID: "759cd68b-755a-47d5-bb83-7a9028adbc86") : object "kube-system"/"original-pull-secret" not registered
Jul 09 02:34:49.678955 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:49.678926    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 02:34:49.972423 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:49.972173    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 02:29:48 +0000 UTC" deadline="2028-04-18 05:28:19.70521063 +0000 UTC"
Jul 09 02:34:49.972576 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:49.972427    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15578h53m29.732790808s"
Jul 09 02:34:50.114420 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:50.113786    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:34:50.114420 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:50.113940    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:34:50.114648 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:50.114504    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:50.114648 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:50.114624    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:34:50.154995 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:50.154956    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-236.ec2.internal" event={"ID":"4d17de08da950bb8f59f5515256e0343","Type":"ContainerStarted","Data":"9eb55328b36951857bd64f892b856565abf7477ad9ccc6692f1bb6c15836509d"}
Jul 09 02:34:50.170079 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:50.170008    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-236.ec2.internal" podStartSLOduration=3.16998915 podStartE2EDuration="3.16998915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 02:34:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:34:50.168951996 +0000 UTC m=+4.562755412" watchObservedRunningTime="2026-07-09 02:34:50.16998915 +0000 UTC m=+4.563792568"
Jul 09 02:34:51.112960 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:51.112390    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:34:51.112960 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:51.112570    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:34:51.165423 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:51.164987    2578 generic.go:356] "Generic (PLEG): container finished" podID="fae37a88080d71f4e174849eeb91a89e" containerID="5bc90f3c356c514d31be6a936d8e562b3c553d1f05f1e6fbd024b94ae0beaf51" exitCode=0
Jul 09 02:34:51.166821 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:51.166673    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal" event={"ID":"fae37a88080d71f4e174849eeb91a89e","Type":"ContainerDied","Data":"5bc90f3c356c514d31be6a936d8e562b3c553d1f05f1e6fbd024b94ae0beaf51"}
Jul 09 02:34:51.588829 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:51.588081    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs\") pod \"network-metrics-daemon-jt6w4\" (UID: \"29ffb437-ae69-4765-85d3-5f1ebe2c0346\") " pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:34:51.588829 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:51.588315    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 02:34:51.588829 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:51.588410    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs podName:29ffb437-ae69-4765-85d3-5f1ebe2c0346 nodeName:}" failed. No retries permitted until 2026-07-09 02:34:55.588386979 +0000 UTC m=+9.982190372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs") pod "network-metrics-daemon-jt6w4" (UID: "29ffb437-ae69-4765-85d3-5f1ebe2c0346") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 02:34:51.689842 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:51.688968    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dwnz\" (UniqueName: \"kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz\") pod \"network-check-target-ppkl8\" (UID: \"f6f3333d-e691-4ae1-8ee6-55f30813e35d\") " pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:34:51.689842 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:51.689022    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret\") pod \"global-pull-secret-syncer-t6lpr\" (UID: \"759cd68b-755a-47d5-bb83-7a9028adbc86\") " pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:51.689842 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:51.689177    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 02:34:51.689842 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:51.689243    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret podName:759cd68b-755a-47d5-bb83-7a9028adbc86 nodeName:}" failed. No retries permitted until 2026-07-09 02:34:55.689224918 +0000 UTC m=+10.083028317 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret") pod "global-pull-secret-syncer-t6lpr" (UID: "759cd68b-755a-47d5-bb83-7a9028adbc86") : object "kube-system"/"original-pull-secret" not registered
Jul 09 02:34:51.689842 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:51.689698    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 02:34:51.689842 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:51.689719    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 02:34:51.689842 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:51.689734    2578 projected.go:196] Error preparing data for projected volume kube-api-access-7dwnz for pod openshift-network-diagnostics/network-check-target-ppkl8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 02:34:51.689842 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:51.689783    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz podName:f6f3333d-e691-4ae1-8ee6-55f30813e35d nodeName:}" failed. No retries permitted until 2026-07-09 02:34:55.689767151 +0000 UTC m=+10.083570547 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7dwnz" (UniqueName: "kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz") pod "network-check-target-ppkl8" (UID: "f6f3333d-e691-4ae1-8ee6-55f30813e35d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 02:34:52.112277 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:52.112231    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:52.112501 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:52.112232    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:34:52.112501 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:52.112409    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:34:52.112501 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:52.112464    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:34:53.111877 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:53.111838    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:34:53.112381 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:53.111996    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:34:54.112449 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:54.112420    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:34:54.112915 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:54.112561    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:34:54.112974 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:54.112931    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:54.113052 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:54.113026    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:34:55.111657 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:55.111619    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:34:55.111822 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:55.111730    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:34:55.621552 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:55.621507    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs\") pod \"network-metrics-daemon-jt6w4\" (UID: \"29ffb437-ae69-4765-85d3-5f1ebe2c0346\") " pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:34:55.621991 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:55.621684    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 02:34:55.621991 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:55.621747    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs podName:29ffb437-ae69-4765-85d3-5f1ebe2c0346 nodeName:}" failed. No retries permitted until 2026-07-09 02:35:03.621726145 +0000 UTC m=+18.015529540 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs") pod "network-metrics-daemon-jt6w4" (UID: "29ffb437-ae69-4765-85d3-5f1ebe2c0346") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 02:34:55.721975 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:55.721932    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dwnz\" (UniqueName: \"kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz\") pod \"network-check-target-ppkl8\" (UID: \"f6f3333d-e691-4ae1-8ee6-55f30813e35d\") " pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:34:55.722156 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:55.721980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret\") pod \"global-pull-secret-syncer-t6lpr\" (UID: \"759cd68b-755a-47d5-bb83-7a9028adbc86\") " pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:55.722156 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:55.722092    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 02:34:55.722156 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:55.722120    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 02:34:55.722156 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:55.722147    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 02:34:55.722156 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:55.722156    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret podName:759cd68b-755a-47d5-bb83-7a9028adbc86 nodeName:}" failed. No retries permitted until 2026-07-09 02:35:03.722137866 +0000 UTC m=+18.115941261 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret") pod "global-pull-secret-syncer-t6lpr" (UID: "759cd68b-755a-47d5-bb83-7a9028adbc86") : object "kube-system"/"original-pull-secret" not registered
Jul 09 02:34:55.722156 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:55.722160    2578 projected.go:196] Error preparing data for projected volume kube-api-access-7dwnz for pod openshift-network-diagnostics/network-check-target-ppkl8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 02:34:55.722424 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:55.722214    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz podName:f6f3333d-e691-4ae1-8ee6-55f30813e35d nodeName:}" failed. No retries permitted until 2026-07-09 02:35:03.722197405 +0000 UTC m=+18.116000820 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7dwnz" (UniqueName: "kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz") pod "network-check-target-ppkl8" (UID: "f6f3333d-e691-4ae1-8ee6-55f30813e35d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 02:34:56.112203 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:56.112170    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:34:56.112398 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:56.112250    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:56.112398 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:56.112271    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:34:56.112398 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:56.112381    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:34:57.112109 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:57.112071    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:34:57.112587 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:57.112203    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:34:58.111741 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:58.111670    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:34:58.111900 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:58.111704    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:34:58.111900 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:58.111850    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:34:58.112030 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:58.111907    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:34:59.112907 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:59.112422    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:34:59.112907 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:34:59.112545    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:34:59.192284 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:59.191597    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal" event={"ID":"fae37a88080d71f4e174849eeb91a89e","Type":"ContainerStarted","Data":"f00d7a9ff259efc0eedd278cffe7ef0c0029b692d721ce0f23d4e3e625b3828f"}
Jul 09 02:34:59.198483 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:59.197472    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4rnpt" event={"ID":"daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6","Type":"ContainerStarted","Data":"766731b373694b2ec07a09351d44f56fa911b07e7f3c2c17d064485aab056a9c"}
Jul 09 02:34:59.206409 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:59.205833    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-236.ec2.internal" podStartSLOduration=12.205812609 podStartE2EDuration="12.205812609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 02:34:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:34:59.205361368 +0000 UTC m=+13.599164782" watchObservedRunningTime="2026-07-09 02:34:59.205812609 +0000 UTC m=+13.599616026"
Jul 09 02:34:59.219251 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:34:59.218643    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4rnpt" podStartSLOduration=2.683350834 podStartE2EDuration="13.218622648s" totalImagesPullingTime="10.535271814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 02:34:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:34:59.21810486 +0000 UTC m=+13.611908273" watchObservedRunningTime="2026-07-09 02:34:59.218622648 +0000 UTC m=+13.612426062"
Jul 09 02:35:00.111976 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:00.111779    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:00.112193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:00.111847    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:00.112193 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:00.112078    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:00.112193 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:00.112144    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:00.201581 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:00.201545    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kxrl5" event={"ID":"147eafc1-fac7-4201-b3d1-261371a5667a","Type":"ContainerStarted","Data":"78c0a4670e9ef40e73a4d5528ca612371b014c6dcd1fc14ab7da2ffe635fb83b"}
Jul 09 02:35:00.203903 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:00.203872    2578 generic.go:356] "Generic (PLEG): container finished" podID="e59810c5-e353-4644-b26c-404664df8c06" containerID="a98ebc3f7867865ee4a5df8227b8b2271b0bf37831a48879483685fdb68f6fc9" exitCode=0
Jul 09 02:35:00.204058 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:00.203944    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8rhkt" event={"ID":"e59810c5-e353-4644-b26c-404664df8c06","Type":"ContainerDied","Data":"a98ebc3f7867865ee4a5df8227b8b2271b0bf37831a48879483685fdb68f6fc9"}
Jul 09 02:35:00.205948 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:00.205917    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x" event={"ID":"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9","Type":"ContainerStarted","Data":"892fbd7c5713f3d759156e2569e99782f48c53cba8f20b98cfd269cd5cb9bdfd"}
Jul 09 02:35:00.207699 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:00.207674    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9qdgf" event={"ID":"a8a39f98-fa17-46a6-b79a-c1d5e1304a39","Type":"ContainerStarted","Data":"d9fa8acebe4ec0a9dc639bd3318fcd9d53d388096971653c7e6465478fe2930c"}
Jul 09 02:35:00.209212 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:00.209186    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pzqxx" event={"ID":"1508f1fe-46da-4bfa-bbd0-a975c19cf63b","Type":"ContainerStarted","Data":"08316f22c3cd79ee2a430b5aa913fe43e32760d3adaf588b1db254a2c83fb574"}
Jul 09 02:35:00.216458 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:00.216408    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kxrl5" podStartSLOduration=2.672110715 podStartE2EDuration="13.216390867s" totalImagesPullingTime="10.544280152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 02:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:00.215946234 +0000 UTC m=+14.609749649" watchObservedRunningTime="2026-07-09 02:35:00.216390867 +0000 UTC m=+14.610194281"
Jul 09 02:35:00.232568 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:00.230760    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9qdgf" podStartSLOduration=3.321269677 podStartE2EDuration="14.230741895s" totalImagesPullingTime="10.909472218s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 02:34:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:00.230113269 +0000 UTC m=+14.623916661" watchObservedRunningTime="2026-07-09 02:35:00.230741895 +0000 UTC m=+14.624545311"
Jul 09 02:35:00.261345 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:00.261283    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pzqxx" podStartSLOduration=3.64380282 podStartE2EDuration="14.26126887s" totalImagesPullingTime="10.61746605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:34:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:00.261130943 +0000 UTC m=+14.654934356" watchObservedRunningTime="2026-07-09 02:35:00.26126887 +0000 UTC m=+14.655072283"
Jul 09 02:35:01.112106 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:01.112071    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:01.112272 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:01.112205    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:01.213222 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:01.213186    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kpjk2" event={"ID":"b55deb49-73ce-4694-9271-21097d8294d1","Type":"ContainerStarted","Data":"a4907ded149c42d17d4261b094122fe2953627d4fac7bbdddb06f79b06dd80bc"}
Jul 09 02:35:02.112299 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:02.112269    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:02.112487 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:02.112441    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:02.112487 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:02.112477    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:02.112606 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:02.112586    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:03.111561 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:03.111524    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:03.112202 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:03.111649    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:03.681571 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:03.681495    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs\") pod \"network-metrics-daemon-jt6w4\" (UID: \"29ffb437-ae69-4765-85d3-5f1ebe2c0346\") " pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:03.681747 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:03.681661    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 02:35:03.681747 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:03.681737    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs podName:29ffb437-ae69-4765-85d3-5f1ebe2c0346 nodeName:}" failed. No retries permitted until 2026-07-09 02:35:19.681719623 +0000 UTC m=+34.075523013 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs") pod "network-metrics-daemon-jt6w4" (UID: "29ffb437-ae69-4765-85d3-5f1ebe2c0346") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 02:35:03.782009 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:03.781964    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dwnz\" (UniqueName: \"kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz\") pod \"network-check-target-ppkl8\" (UID: \"f6f3333d-e691-4ae1-8ee6-55f30813e35d\") " pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:03.782243 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:03.782022    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret\") pod \"global-pull-secret-syncer-t6lpr\" (UID: \"759cd68b-755a-47d5-bb83-7a9028adbc86\") " pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:03.782243 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:03.782142    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 02:35:03.782243 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:03.782157    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 02:35:03.782243 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:03.782168    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 02:35:03.782243 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:03.782182    2578 projected.go:196] Error preparing data for projected volume kube-api-access-7dwnz for pod openshift-network-diagnostics/network-check-target-ppkl8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 02:35:03.782243 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:03.782222    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret podName:759cd68b-755a-47d5-bb83-7a9028adbc86 nodeName:}" failed. No retries permitted until 2026-07-09 02:35:19.782204224 +0000 UTC m=+34.176007622 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret") pod "global-pull-secret-syncer-t6lpr" (UID: "759cd68b-755a-47d5-bb83-7a9028adbc86") : object "kube-system"/"original-pull-secret" not registered
Jul 09 02:35:03.782243 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:03.782241    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz podName:f6f3333d-e691-4ae1-8ee6-55f30813e35d nodeName:}" failed. No retries permitted until 2026-07-09 02:35:19.782231404 +0000 UTC m=+34.176034799 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7dwnz" (UniqueName: "kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz") pod "network-check-target-ppkl8" (UID: "f6f3333d-e691-4ae1-8ee6-55f30813e35d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 02:35:04.112485 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:04.112402    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:04.112911 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:04.112404    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:04.112911 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:04.112521    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:04.112911 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:04.112578    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:04.843427 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:04.843389    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pzqxx"
Jul 09 02:35:04.844742 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:04.844717    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pzqxx"
Jul 09 02:35:04.858247 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:04.858197    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kpjk2" podStartSLOduration=8.282819839 podStartE2EDuration="18.85818245s" totalImagesPullingTime="10.575362611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 02:34:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:01.227381076 +0000 UTC m=+15.621184487" watchObservedRunningTime="2026-07-09 02:35:04.85818245 +0000 UTC m=+19.251985862"
Jul 09 02:35:05.111525 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:05.111443    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:05.111708 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:05.111569    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:06.113483 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:06.113444    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:06.113912 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:06.113573    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:06.113912 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:06.113655    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:06.113912 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:06.113768    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:07.112211 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:07.112182    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:07.112400 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:07.112288    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:08.112069 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:08.112020    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:08.112487 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:08.112163    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:08.112487 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:08.112241    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:08.112487 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:08.112370    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:08.491677 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:08.491656    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 02:35:09.020555 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:09.020234    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T02:35:08.491674024Z","UUID":"e3d71ee6-8bc1-45cb-9f94-9948cfa215fa","Handler":null,"Name":"","Endpoint":""}
Jul 09 02:35:09.022258 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:09.022135    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 02:35:09.022258 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:09.022182    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 02:35:09.112505 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:09.112476    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:09.113104 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:09.112598    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:09.232388 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:09.232277    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5jtrv" event={"ID":"77446bdb-57d3-4db7-8518-63cdb3111bd9","Type":"ContainerStarted","Data":"955baa29d9f7ebb08fa130888ca80cc4ccb0ae9f23b56c4d9d8b871e629ada22"}
Jul 09 02:35:09.234039 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:09.234009    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x" event={"ID":"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9","Type":"ContainerStarted","Data":"620994d2972503ab7dcfba4a115e2f6b2362466fed30172d45f69f3d83924218"}
Jul 09 02:35:09.236737 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:09.236706    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz" event={"ID":"d33c8ed2-57be-4728-bf4e-52cc94d3e001","Type":"ContainerStarted","Data":"78d15b3e20c804aec4c31e20548c7e49241ae13c7813cd8c30f76f89dffb0efc"}
Jul 09 02:35:09.236846 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:09.236773    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz" event={"ID":"d33c8ed2-57be-4728-bf4e-52cc94d3e001","Type":"ContainerStarted","Data":"d0cb43f19f6d2569ab4efc21374bf958a883beb149f738004384ffbbd502b45c"}
Jul 09 02:35:09.236846 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:09.236803    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz" event={"ID":"d33c8ed2-57be-4728-bf4e-52cc94d3e001","Type":"ContainerStarted","Data":"b081cd1e4addddc77cf94bd8776621a4f3e47d7547c93f4aa013fe4becec5415"}
Jul 09 02:35:09.236846 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:09.236832    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz" event={"ID":"d33c8ed2-57be-4728-bf4e-52cc94d3e001","Type":"ContainerStarted","Data":"ea5f93204373abfee8977cae0dce0041ba5e3a3db389d1c6c9648ce9bf8a6642"}
Jul 09 02:35:09.236946 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:09.236857    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz" event={"ID":"d33c8ed2-57be-4728-bf4e-52cc94d3e001","Type":"ContainerStarted","Data":"5bf4591ef26e48e292e047f764dee1ef7a9492b901d426a36aa2310a9e4241d5"}
Jul 09 02:35:09.236946 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:09.236884    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz" event={"ID":"d33c8ed2-57be-4728-bf4e-52cc94d3e001","Type":"ContainerStarted","Data":"c794922d98e20f64145bc95419f3f857bb1dc3d9c02c65e8a5a31e47ba52b67d"}
Jul 09 02:35:09.253746 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:09.252224    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5jtrv" podStartSLOduration=2.295353762 podStartE2EDuration="22.2522039s" totalImagesPullingTime="19.956850138s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 02:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:09.251411546 +0000 UTC m=+23.645214962" watchObservedRunningTime="2026-07-09 02:35:09.2522039 +0000 UTC m=+23.646007318"
Jul 09 02:35:10.112249 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:10.112179    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:10.112455 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:10.112298    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:10.112455 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:10.112349    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:10.112455 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:10.112431    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:10.240524 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:10.240490    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x" event={"ID":"3935cfd9-d4ef-4a7d-8f7d-36b78e7b0cd9","Type":"ContainerStarted","Data":"98df784ce089734902228e908666e29b32d413afc3bbc94a670b77d1c7a61212"}
Jul 09 02:35:10.256781 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:10.256741    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2bc7x" podStartSLOduration=3.478510672 podStartE2EDuration="24.256727071s" totalImagesPullingTime="20.778216399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 02:34:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:10.256649215 +0000 UTC m=+24.650452618" watchObservedRunningTime="2026-07-09 02:35:10.256727071 +0000 UTC m=+24.650530480"
Jul 09 02:35:11.111741 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:11.111714    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:11.111850 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:11.111827    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:11.247491 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:11.247450    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz" event={"ID":"d33c8ed2-57be-4728-bf4e-52cc94d3e001","Type":"ContainerStarted","Data":"f5977288a08e2ed9d061d9fab503831629e2d4f6fcf28809d081dd31c1573a38"}
Jul 09 02:35:11.729546 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:11.729518    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4rnpt_daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6/dns-node-resolver/0.log"
Jul 09 02:35:12.111644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:12.111573    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:12.111644 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:12.111594    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:12.111840 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:12.111689    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:12.111878 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:12.111837    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:12.786030 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:12.785996    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pzqxx"
Jul 09 02:35:12.786526 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:12.786164    2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 09 02:35:12.786706 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:12.786686    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pzqxx"
Jul 09 02:35:12.902415 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:12.902381    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kxrl5_147eafc1-fac7-4201-b3d1-261371a5667a/node-ca/0.log"
Jul 09 02:35:13.111501 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:13.111422    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:13.111665 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:13.111546    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:14.112585 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:14.112431    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:14.113034 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:14.112435    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:14.113034 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:14.112720    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:14.113034 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:14.112764    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:14.257956 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:14.257922    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz" event={"ID":"d33c8ed2-57be-4728-bf4e-52cc94d3e001","Type":"ContainerStarted","Data":"fe7babb1144f5be71676676711896b5f23d777758656e5d9fa2b7b81556355a1"}
Jul 09 02:35:14.258197 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:14.258176    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:35:14.258308 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:14.258287    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:35:14.260035 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:14.260008    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8rhkt" event={"ID":"e59810c5-e353-4644-b26c-404664df8c06","Type":"ContainerStarted","Data":"94f98e0fbae4e710fee000a1c227b14802932adda88f6bce754e51b787972afe"}
Jul 09 02:35:14.273850 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:14.273825    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:35:14.285697 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:14.285660    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz" podStartSLOduration=8.140443198 podStartE2EDuration="28.285647782s" totalImagesPullingTime="20.145204584s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 02:34:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:14.285192529 +0000 UTC m=+28.678995941" watchObservedRunningTime="2026-07-09 02:35:14.285647782 +0000 UTC m=+28.679451194"
Jul 09 02:35:15.111687 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:15.111658    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:15.111914 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:15.111747    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:15.263712 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:15.263679    2578 generic.go:356] "Generic (PLEG): container finished" podID="e59810c5-e353-4644-b26c-404664df8c06" containerID="94f98e0fbae4e710fee000a1c227b14802932adda88f6bce754e51b787972afe" exitCode=0
Jul 09 02:35:15.264087 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:15.263762    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8rhkt" event={"ID":"e59810c5-e353-4644-b26c-404664df8c06","Type":"ContainerDied","Data":"94f98e0fbae4e710fee000a1c227b14802932adda88f6bce754e51b787972afe"}
Jul 09 02:35:15.264295 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:15.264276    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:35:15.279376 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:15.279359    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:35:16.113140 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:16.113111    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:16.113293 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:16.113192    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:16.113368 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:16.113299    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:16.113464 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:16.113441    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:17.111902 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:17.111881    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:17.112153 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:17.111980    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:17.270803 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:17.270774    2578 generic.go:356] "Generic (PLEG): container finished" podID="e59810c5-e353-4644-b26c-404664df8c06" containerID="6453bc8368259074955a0859450ab3716a54f0896c3c3258fba4ee6e1ad026ba" exitCode=0
Jul 09 02:35:17.270907 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:17.270857    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8rhkt" event={"ID":"e59810c5-e353-4644-b26c-404664df8c06","Type":"ContainerDied","Data":"6453bc8368259074955a0859450ab3716a54f0896c3c3258fba4ee6e1ad026ba"}
Jul 09 02:35:18.111497 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:18.111463    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:18.111675 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:18.111463    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:18.111675 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:18.111563    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:18.111773 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:18.111671    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:19.111926 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:19.111894    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:19.112438 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:19.112026    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:19.276950 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:19.276912    2578 generic.go:356] "Generic (PLEG): container finished" podID="e59810c5-e353-4644-b26c-404664df8c06" containerID="616c4c01cad9589836c0d5506ba98bd834f99303c6d9a2f818f4a36193fc79a1" exitCode=0
Jul 09 02:35:19.277114 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:19.276964    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8rhkt" event={"ID":"e59810c5-e353-4644-b26c-404664df8c06","Type":"ContainerDied","Data":"616c4c01cad9589836c0d5506ba98bd834f99303c6d9a2f818f4a36193fc79a1"}
Jul 09 02:35:19.708695 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:19.708669    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs\") pod \"network-metrics-daemon-jt6w4\" (UID: \"29ffb437-ae69-4765-85d3-5f1ebe2c0346\") " pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:19.708848 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:19.708781    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 02:35:19.708848 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:19.708833    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs podName:29ffb437-ae69-4765-85d3-5f1ebe2c0346 nodeName:}" failed. No retries permitted until 2026-07-09 02:35:51.70881499 +0000 UTC m=+66.102618400 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs") pod "network-metrics-daemon-jt6w4" (UID: "29ffb437-ae69-4765-85d3-5f1ebe2c0346") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 02:35:19.808934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:19.808904    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dwnz\" (UniqueName: \"kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz\") pod \"network-check-target-ppkl8\" (UID: \"f6f3333d-e691-4ae1-8ee6-55f30813e35d\") " pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:19.809068 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:19.808937    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret\") pod \"global-pull-secret-syncer-t6lpr\" (UID: \"759cd68b-755a-47d5-bb83-7a9028adbc86\") " pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:19.809068 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:19.809035    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 02:35:19.809068 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:19.809050    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 02:35:19.809160 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:19.809071    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 02:35:19.809160 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:19.809082    2578 projected.go:196] Error preparing data for projected volume kube-api-access-7dwnz for pod openshift-network-diagnostics/network-check-target-ppkl8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 02:35:19.809160 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:19.809101    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret podName:759cd68b-755a-47d5-bb83-7a9028adbc86 nodeName:}" failed. No retries permitted until 2026-07-09 02:35:51.809066858 +0000 UTC m=+66.202870249 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret") pod "global-pull-secret-syncer-t6lpr" (UID: "759cd68b-755a-47d5-bb83-7a9028adbc86") : object "kube-system"/"original-pull-secret" not registered
Jul 09 02:35:19.809160 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:19.809118    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz podName:f6f3333d-e691-4ae1-8ee6-55f30813e35d nodeName:}" failed. No retries permitted until 2026-07-09 02:35:51.809107857 +0000 UTC m=+66.202911248 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7dwnz" (UniqueName: "kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz") pod "network-check-target-ppkl8" (UID: "f6f3333d-e691-4ae1-8ee6-55f30813e35d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 02:35:20.111591 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:20.111523    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:20.111734 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:20.111526    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:20.111734 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:20.111617    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:20.111734 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:20.111683    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:21.111472 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:21.111446    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:21.111995 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:21.111566    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:22.112138 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:22.112108    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:22.112645 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:22.112107    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:22.112645 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:22.112251    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:22.112645 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:22.112332    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:23.112014 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:23.111833    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:23.112182 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:23.112092    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:24.111957 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:24.111921    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:24.112141 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:24.111927    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:24.112141 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:24.112075    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:24.112141 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:24.112119    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:25.112406 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:25.112376    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:25.112764 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:25.112464    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:26.112774 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:26.112738    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:26.113217 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:26.112824    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:26.113217 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:26.112908    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:26.113217 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:26.112993    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:26.295430 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:26.295395    2578 generic.go:356] "Generic (PLEG): container finished" podID="e59810c5-e353-4644-b26c-404664df8c06" containerID="18d4675d369d097ef398d1f506f80a17f674e28d5e718b3018c7ab231df5c814" exitCode=0
Jul 09 02:35:26.295594 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:26.295433    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8rhkt" event={"ID":"e59810c5-e353-4644-b26c-404664df8c06","Type":"ContainerDied","Data":"18d4675d369d097ef398d1f506f80a17f674e28d5e718b3018c7ab231df5c814"}
Jul 09 02:35:27.112032 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:27.112005    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:27.112201 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:27.112096    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:27.300366 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:27.300332    2578 generic.go:356] "Generic (PLEG): container finished" podID="e59810c5-e353-4644-b26c-404664df8c06" containerID="5a49be09a534c5998bfd49dc38cf70495ea4d7612c5e9b72944d3d253c99154d" exitCode=0
Jul 09 02:35:27.300755 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:27.300383    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8rhkt" event={"ID":"e59810c5-e353-4644-b26c-404664df8c06","Type":"ContainerDied","Data":"5a49be09a534c5998bfd49dc38cf70495ea4d7612c5e9b72944d3d253c99154d"}
Jul 09 02:35:28.111803 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:28.111773    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:28.111958 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:28.111893    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:28.111958 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:28.111947    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:28.112046 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:28.112033    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:28.305459 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:28.305423    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8rhkt" event={"ID":"e59810c5-e353-4644-b26c-404664df8c06","Type":"ContainerStarted","Data":"3cf68526b7660cb9b22dff5376d122d00e83867512b5a5c22982b69bfc87ab22"}
Jul 09 02:35:28.327878 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:28.327836    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8rhkt" podStartSLOduration=8.976379751 podStartE2EDuration="42.327818908s" totalImagesPullingTime="33.351439157s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 02:34:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:28.326911276 +0000 UTC m=+42.720714689" watchObservedRunningTime="2026-07-09 02:35:28.327818908 +0000 UTC m=+42.721622325"
Jul 09 02:35:29.112389 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:29.112360    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:29.112554 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:29.112454    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:30.112413 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:30.112376    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:30.112413 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:30.112400    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:30.112809 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:30.112546    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:30.112809 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:30.112616    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:31.111501 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:31.111472    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:31.111659 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:31.111563    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:32.112214 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:32.112183    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:32.112621 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:32.112347    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:32.112621 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:32.112392    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:32.112621 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:32.112495    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:33.112109 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:33.112083    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:33.112274 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:33.112174    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:34.112394 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:34.112364    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:34.112752 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:34.112469    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:34.112752 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:34.112499    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:34.112752 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:34.112566    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:35.112013 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:35.111986    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:35.112170 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:35.112075    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:36.112507 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:36.112479    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:36.112846 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:36.112561    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:36.112846 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:36.112646    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:36.112846 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:36.112737    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:37.111363 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:37.111334    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:37.111617 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:37.111452    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:38.026618 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:38.026579    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ppkl8"]
Jul 09 02:35:38.027485 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:38.027464    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jt6w4"]
Jul 09 02:35:38.027622 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:38.027550    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:38.027684 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:38.027650    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:38.027813 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:38.027791    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:38.028004 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:38.027972    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:38.028709 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:38.028691    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t6lpr"]
Jul 09 02:35:38.028806 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:38.028770    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:38.028867 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:38.028831    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:40.111845 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:40.111812    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:40.111845 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:40.111823    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:40.112285 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:40.111923    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:40.112285 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:40.111973    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:40.112285 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:40.112014    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:40.112285 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:40.112073    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:42.112409 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.112212    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:42.112772 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.112215    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:42.112772 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:42.112491    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jt6w4" podUID="29ffb437-ae69-4765-85d3-5f1ebe2c0346"
Jul 09 02:35:42.112772 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:42.112551    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ppkl8" podUID="f6f3333d-e691-4ae1-8ee6-55f30813e35d"
Jul 09 02:35:42.112772 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.112213    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:42.112772 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:42.112619    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t6lpr" podUID="759cd68b-755a-47d5-bb83-7a9028adbc86"
Jul 09 02:35:42.883707 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.883684    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-236.ec2.internal" event="NodeReady"
Jul 09 02:35:42.883830 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.883779    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 02:35:42.927207 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.927183    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6b29q"]
Jul 09 02:35:42.928859 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.928845    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6b29q"
Jul 09 02:35:42.930874 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.930852    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 02:35:42.930874 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.930867    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-df9xh\""
Jul 09 02:35:42.931186 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.931171    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 02:35:42.931237 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.931198    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 02:35:42.933610 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.933594    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zc8cn"]
Jul 09 02:35:42.935418 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.935393    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:42.937563 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.937546    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 02:35:42.937724 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.937706    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 02:35:42.937777 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.937760    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 02:35:42.937830 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.937764    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xhqgt\""
Jul 09 02:35:42.937830 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.937801    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 02:35:42.939863 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.939844    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6b29q"]
Jul 09 02:35:42.945708 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.945690    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zc8cn"]
Jul 09 02:35:42.948989 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.948968    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-j99cn"]
Jul 09 02:35:42.950790 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.950775    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j99cn"
Jul 09 02:35:42.953181 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.953162    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 02:35:42.953271 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.953215    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jllr5\""
Jul 09 02:35:42.953340 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.953273    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 02:35:42.963662 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.963644    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j99cn"]
Jul 09 02:35:42.981053 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.981033    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa-tmp-dir\") pod \"dns-default-j99cn\" (UID: \"d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa\") " pod="openshift-dns/dns-default-j99cn"
Jul 09 02:35:42.981145 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.981056    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b76c7f8-c7e2-48dd-a392-90409477fb13-data-volume\") pod \"insights-runtime-extractor-zc8cn\" (UID: \"9b76c7f8-c7e2-48dd-a392-90409477fb13\") " pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:42.981145 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.981074    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2415615-0367-4de5-b7f8-afd1945d4b4e-cert\") pod \"ingress-canary-6b29q\" (UID: \"c2415615-0367-4de5-b7f8-afd1945d4b4e\") " pod="openshift-ingress-canary/ingress-canary-6b29q"
Jul 09 02:35:42.981145 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.981093    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9b76c7f8-c7e2-48dd-a392-90409477fb13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zc8cn\" (UID: \"9b76c7f8-c7e2-48dd-a392-90409477fb13\") " pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:42.981145 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.981126    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g75lt\" (UniqueName: \"kubernetes.io/projected/c2415615-0367-4de5-b7f8-afd1945d4b4e-kube-api-access-g75lt\") pod \"ingress-canary-6b29q\" (UID: \"c2415615-0367-4de5-b7f8-afd1945d4b4e\") " pod="openshift-ingress-canary/ingress-canary-6b29q"
Jul 09 02:35:42.981308 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.981171    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa-config-volume\") pod \"dns-default-j99cn\" (UID: \"d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa\") " pod="openshift-dns/dns-default-j99cn"
Jul 09 02:35:42.981308 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.981207    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa-metrics-tls\") pod \"dns-default-j99cn\" (UID: \"d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa\") " pod="openshift-dns/dns-default-j99cn"
Jul 09 02:35:42.981308 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.981224    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stnsl\" (UniqueName: \"kubernetes.io/projected/9b76c7f8-c7e2-48dd-a392-90409477fb13-kube-api-access-stnsl\") pod \"insights-runtime-extractor-zc8cn\" (UID: \"9b76c7f8-c7e2-48dd-a392-90409477fb13\") " pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:42.981308 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.981243    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9b76c7f8-c7e2-48dd-a392-90409477fb13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zc8cn\" (UID: \"9b76c7f8-c7e2-48dd-a392-90409477fb13\") " pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:42.981308 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.981287    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq6pq\" (UniqueName: \"kubernetes.io/projected/d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa-kube-api-access-fq6pq\") pod \"dns-default-j99cn\" (UID: \"d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa\") " pod="openshift-dns/dns-default-j99cn"
Jul 09 02:35:42.981479 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:42.981311    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9b76c7f8-c7e2-48dd-a392-90409477fb13-crio-socket\") pod \"insights-runtime-extractor-zc8cn\" (UID: \"9b76c7f8-c7e2-48dd-a392-90409477fb13\") " pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:43.081906 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.081816    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa-tmp-dir\") pod \"dns-default-j99cn\" (UID: \"d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa\") " pod="openshift-dns/dns-default-j99cn"
Jul 09 02:35:43.081906 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.081853    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b76c7f8-c7e2-48dd-a392-90409477fb13-data-volume\") pod \"insights-runtime-extractor-zc8cn\" (UID: \"9b76c7f8-c7e2-48dd-a392-90409477fb13\") " pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:43.081906 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.081880    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2415615-0367-4de5-b7f8-afd1945d4b4e-cert\") pod \"ingress-canary-6b29q\" (UID: \"c2415615-0367-4de5-b7f8-afd1945d4b4e\") " pod="openshift-ingress-canary/ingress-canary-6b29q"
Jul 09 02:35:43.082105 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.081906    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9b76c7f8-c7e2-48dd-a392-90409477fb13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zc8cn\" (UID: \"9b76c7f8-c7e2-48dd-a392-90409477fb13\") " pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:43.082105 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.081948    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g75lt\" (UniqueName: \"kubernetes.io/projected/c2415615-0367-4de5-b7f8-afd1945d4b4e-kube-api-access-g75lt\") pod \"ingress-canary-6b29q\" (UID: \"c2415615-0367-4de5-b7f8-afd1945d4b4e\") " pod="openshift-ingress-canary/ingress-canary-6b29q"
Jul 09 02:35:43.082105 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.081982    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa-config-volume\") pod \"dns-default-j99cn\" (UID: \"d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa\") " pod="openshift-dns/dns-default-j99cn"
Jul 09 02:35:43.082105 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.082021    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa-metrics-tls\") pod \"dns-default-j99cn\" (UID: \"d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa\") " pod="openshift-dns/dns-default-j99cn"
Jul 09 02:35:43.082105 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.082045    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stnsl\" (UniqueName: \"kubernetes.io/projected/9b76c7f8-c7e2-48dd-a392-90409477fb13-kube-api-access-stnsl\") pod \"insights-runtime-extractor-zc8cn\" (UID: \"9b76c7f8-c7e2-48dd-a392-90409477fb13\") " pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:43.082105 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.082075    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9b76c7f8-c7e2-48dd-a392-90409477fb13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zc8cn\" (UID: \"9b76c7f8-c7e2-48dd-a392-90409477fb13\") " pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:43.082390 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.082116    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq6pq\" (UniqueName: \"kubernetes.io/projected/d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa-kube-api-access-fq6pq\") pod \"dns-default-j99cn\" (UID: \"d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa\") " pod="openshift-dns/dns-default-j99cn"
Jul 09 02:35:43.082390 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.082141    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9b76c7f8-c7e2-48dd-a392-90409477fb13-crio-socket\") pod \"insights-runtime-extractor-zc8cn\" (UID: \"9b76c7f8-c7e2-48dd-a392-90409477fb13\") " pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:43.082390 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.082219    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa-tmp-dir\") pod \"dns-default-j99cn\" (UID: \"d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa\") " pod="openshift-dns/dns-default-j99cn"
Jul 09 02:35:43.082390 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.082227    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b76c7f8-c7e2-48dd-a392-90409477fb13-data-volume\") pod \"insights-runtime-extractor-zc8cn\" (UID: \"9b76c7f8-c7e2-48dd-a392-90409477fb13\") " pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:43.082390 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.082347    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9b76c7f8-c7e2-48dd-a392-90409477fb13-crio-socket\") pod \"insights-runtime-extractor-zc8cn\" (UID: \"9b76c7f8-c7e2-48dd-a392-90409477fb13\") " pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:43.082634 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.082616    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa-config-volume\") pod \"dns-default-j99cn\" (UID: \"d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa\") " pod="openshift-dns/dns-default-j99cn"
Jul 09 02:35:43.082788 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.082770    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9b76c7f8-c7e2-48dd-a392-90409477fb13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zc8cn\" (UID: \"9b76c7f8-c7e2-48dd-a392-90409477fb13\") " pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:43.085769 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.085748    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa-metrics-tls\") pod \"dns-default-j99cn\" (UID: \"d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa\") " pod="openshift-dns/dns-default-j99cn"
Jul 09 02:35:43.085903 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.085883    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9b76c7f8-c7e2-48dd-a392-90409477fb13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zc8cn\" (UID: \"9b76c7f8-c7e2-48dd-a392-90409477fb13\") " pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:43.085996 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.085979    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2415615-0367-4de5-b7f8-afd1945d4b4e-cert\") pod \"ingress-canary-6b29q\" (UID: \"c2415615-0367-4de5-b7f8-afd1945d4b4e\") " pod="openshift-ingress-canary/ingress-canary-6b29q"
Jul 09 02:35:43.089695 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.089672    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g75lt\" (UniqueName: \"kubernetes.io/projected/c2415615-0367-4de5-b7f8-afd1945d4b4e-kube-api-access-g75lt\") pod \"ingress-canary-6b29q\" (UID: \"c2415615-0367-4de5-b7f8-afd1945d4b4e\") " pod="openshift-ingress-canary/ingress-canary-6b29q"
Jul 09 02:35:43.089869 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.089850    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stnsl\" (UniqueName: \"kubernetes.io/projected/9b76c7f8-c7e2-48dd-a392-90409477fb13-kube-api-access-stnsl\") pod \"insights-runtime-extractor-zc8cn\" (UID: \"9b76c7f8-c7e2-48dd-a392-90409477fb13\") " pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:43.089910 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.089850    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq6pq\" (UniqueName: \"kubernetes.io/projected/d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa-kube-api-access-fq6pq\") pod \"dns-default-j99cn\" (UID: \"d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa\") " pod="openshift-dns/dns-default-j99cn"
Jul 09 02:35:43.238677 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.238654    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6b29q"
Jul 09 02:35:43.245654 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.245636    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zc8cn"
Jul 09 02:35:43.258916 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.258887    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j99cn"
Jul 09 02:35:43.392501 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.392472    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zc8cn"]
Jul 09 02:35:43.395699 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.395674    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6b29q"]
Jul 09 02:35:43.395994 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:35:43.395972    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b76c7f8_c7e2_48dd_a392_90409477fb13.slice/crio-47263f7eed99b8320ed2521ba532c43ad30fd52e9a240a8741f67e633d412cdd WatchSource:0}: Error finding container 47263f7eed99b8320ed2521ba532c43ad30fd52e9a240a8741f67e633d412cdd: Status 404 returned error can't find the container with id 47263f7eed99b8320ed2521ba532c43ad30fd52e9a240a8741f67e633d412cdd
Jul 09 02:35:43.399606 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:35:43.399581    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2415615_0367_4de5_b7f8_afd1945d4b4e.slice/crio-b3bdf9b6b56e7306a6abf2388c20be07e0f71a2abc0979dcfdc24bdad33bf790 WatchSource:0}: Error finding container b3bdf9b6b56e7306a6abf2388c20be07e0f71a2abc0979dcfdc24bdad33bf790: Status 404 returned error can't find the container with id b3bdf9b6b56e7306a6abf2388c20be07e0f71a2abc0979dcfdc24bdad33bf790
Jul 09 02:35:43.401635 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:43.400863    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j99cn"]
Jul 09 02:35:43.405293 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:35:43.405275    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7b28cf3_0900_48a4_a3a2_27e4ef3d72aa.slice/crio-472b81f48f56a2e4ef3d98c0b10c7179048c4cf579d4134a82b48dd4cd26bb87 WatchSource:0}: Error finding container 472b81f48f56a2e4ef3d98c0b10c7179048c4cf579d4134a82b48dd4cd26bb87: Status 404 returned error can't find the container with id 472b81f48f56a2e4ef3d98c0b10c7179048c4cf579d4134a82b48dd4cd26bb87
Jul 09 02:35:44.113091 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:44.113055    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:44.114259 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:44.113813    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:44.114259 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:44.113821    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:44.115704 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:44.115279    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 02:35:44.115704 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:44.115298    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 02:35:44.115704 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:44.115578    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9ptb2\""
Jul 09 02:35:44.116003 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:44.115987    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 02:35:44.116244 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:44.116226    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w8vs7\""
Jul 09 02:35:44.116466 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:44.116451    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 02:35:44.344974 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:44.344874    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zc8cn" event={"ID":"9b76c7f8-c7e2-48dd-a392-90409477fb13","Type":"ContainerStarted","Data":"47263f7eed99b8320ed2521ba532c43ad30fd52e9a240a8741f67e633d412cdd"}
Jul 09 02:35:44.346956 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:44.346928    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6b29q" event={"ID":"c2415615-0367-4de5-b7f8-afd1945d4b4e","Type":"ContainerStarted","Data":"b3bdf9b6b56e7306a6abf2388c20be07e0f71a2abc0979dcfdc24bdad33bf790"}
Jul 09 02:35:44.348600 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:44.348532    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j99cn" event={"ID":"d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa","Type":"ContainerStarted","Data":"472b81f48f56a2e4ef3d98c0b10c7179048c4cf579d4134a82b48dd4cd26bb87"}
Jul 09 02:35:46.839798 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.839591    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"]
Jul 09 02:35:46.844457 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.844430    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"
Jul 09 02:35:46.846632 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.846596    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 09 02:35:46.846770 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.846599    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 09 02:35:46.846770 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.846599    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 02:35:46.847048 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.847031    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 02:35:46.847128 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.847057    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 02:35:46.847176 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.847035    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-ggblg\""
Jul 09 02:35:46.852907 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.852595    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"]
Jul 09 02:35:46.859936 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.859789    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kplfz"]
Jul 09 02:35:46.865904 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.864858    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"]
Jul 09 02:35:46.866010 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.865972    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:46.868290 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.867966    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 02:35:46.868290 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.867988    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 02:35:46.868290 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.868019    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:46.868290 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.868034    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-w9bpf\""
Jul 09 02:35:46.869090 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.869072    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 02:35:46.870711 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.870544    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 09 02:35:46.870711 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.870599    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-xqml6\""
Jul 09 02:35:46.870711 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.870704    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 09 02:35:46.870906 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.870766    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 09 02:35:46.874832 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.874810    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"]
Jul 09 02:35:46.914947 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.914915    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e15e0a7-3794-464b-8af7-87ec9972b022-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:46.914947 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.914947    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgt62\" (UniqueName: \"kubernetes.io/projected/47f5a573-8ada-41c9-b55d-67c511698be5-kube-api-access-jgt62\") pod \"openshift-state-metrics-54cf96dfcf-c5872\" (UID: \"47f5a573-8ada-41c9-b55d-67c511698be5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"
Jul 09 02:35:46.915144 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.914984    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e15e0a7-3794-464b-8af7-87ec9972b022-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:46.915144 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.915011    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:46.915144 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.915039    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f5a573-8ada-41c9-b55d-67c511698be5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-c5872\" (UID: \"47f5a573-8ada-41c9-b55d-67c511698be5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"
Jul 09 02:35:46.915144 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.915065    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-textfile\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:46.915144 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.915117    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47f5a573-8ada-41c9-b55d-67c511698be5-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-c5872\" (UID: \"47f5a573-8ada-41c9-b55d-67c511698be5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"
Jul 09 02:35:46.915144 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.915140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e15e0a7-3794-464b-8af7-87ec9972b022-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:46.915424 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.915157    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-accelerators-collector-config\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:46.915424 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.915189    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxtkx\" (UniqueName: \"kubernetes.io/projected/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-kube-api-access-jxtkx\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:46.915424 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.915216    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6e15e0a7-3794-464b-8af7-87ec9972b022-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:46.915424 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.915263    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47f5a573-8ada-41c9-b55d-67c511698be5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-c5872\" (UID: \"47f5a573-8ada-41c9-b55d-67c511698be5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"
Jul 09 02:35:46.915424 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.915338    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-tls\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:46.915682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.915413    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57bm2\" (UniqueName: \"kubernetes.io/projected/6e15e0a7-3794-464b-8af7-87ec9972b022-kube-api-access-57bm2\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:46.915682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.915466    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6e15e0a7-3794-464b-8af7-87ec9972b022-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:46.915682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.915487    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-sys\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:46.915682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.915512    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-root\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:46.915682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.915534    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-metrics-client-ca\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:46.915682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:46.915605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-wtmp\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.016969 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.016937    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57bm2\" (UniqueName: \"kubernetes.io/projected/6e15e0a7-3794-464b-8af7-87ec9972b022-kube-api-access-57bm2\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:47.017128 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.016980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6e15e0a7-3794-464b-8af7-87ec9972b022-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:47.017128 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017007    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-sys\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.017128 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017066    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-sys\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.017128 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017081    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-root\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.017128 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017123    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-metrics-client-ca\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.017415 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017143    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-wtmp\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.017415 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017181    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-root\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.017415 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017187    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e15e0a7-3794-464b-8af7-87ec9972b022-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:47.017415 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgt62\" (UniqueName: \"kubernetes.io/projected/47f5a573-8ada-41c9-b55d-67c511698be5-kube-api-access-jgt62\") pod \"openshift-state-metrics-54cf96dfcf-c5872\" (UID: \"47f5a573-8ada-41c9-b55d-67c511698be5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"
Jul 09 02:35:47.017415 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017270    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e15e0a7-3794-464b-8af7-87ec9972b022-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:47.017415 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:47.017294    2578 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 09 02:35:47.017415 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017300    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.017415 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017351    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f5a573-8ada-41c9-b55d-67c511698be5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-c5872\" (UID: \"47f5a573-8ada-41c9-b55d-67c511698be5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"
Jul 09 02:35:47.017415 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017376    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-textfile\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.017415 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017391    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-wtmp\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.017415 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:47.017404    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e15e0a7-3794-464b-8af7-87ec9972b022-kube-state-metrics-tls podName:6e15e0a7-3794-464b-8af7-87ec9972b022 nodeName:}" failed. No retries permitted until 2026-07-09 02:35:47.517382464 +0000 UTC m=+61.911185869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/6e15e0a7-3794-464b-8af7-87ec9972b022-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-zlnbl" (UID: "6e15e0a7-3794-464b-8af7-87ec9972b022") : secret "kube-state-metrics-tls" not found
Jul 09 02:35:47.017975 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017472    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47f5a573-8ada-41c9-b55d-67c511698be5-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-c5872\" (UID: \"47f5a573-8ada-41c9-b55d-67c511698be5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"
Jul 09 02:35:47.017975 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017503    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e15e0a7-3794-464b-8af7-87ec9972b022-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:47.017975 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017531    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-accelerators-collector-config\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.017975 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017579    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxtkx\" (UniqueName: \"kubernetes.io/projected/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-kube-api-access-jxtkx\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.017975 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017605    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6e15e0a7-3794-464b-8af7-87ec9972b022-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:47.017975 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017627    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47f5a573-8ada-41c9-b55d-67c511698be5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-c5872\" (UID: \"47f5a573-8ada-41c9-b55d-67c511698be5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"
Jul 09 02:35:47.017975 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017647    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-textfile\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.017975 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017655    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-tls\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.017975 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:47.017726    2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 09 02:35:47.017975 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:35:47.017771    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-tls podName:2c5d0295-77c6-4ea3-9826-a7978bbfb31c nodeName:}" failed. No retries permitted until 2026-07-09 02:35:47.517754194 +0000 UTC m=+61.911557585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-tls") pod "node-exporter-kplfz" (UID: "2c5d0295-77c6-4ea3-9826-a7978bbfb31c") : secret "node-exporter-tls" not found
Jul 09 02:35:47.017975 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017808    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-metrics-client-ca\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.017975 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.017844    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6e15e0a7-3794-464b-8af7-87ec9972b022-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:47.018575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.018106    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6e15e0a7-3794-464b-8af7-87ec9972b022-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:47.018575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.018174    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47f5a573-8ada-41c9-b55d-67c511698be5-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-c5872\" (UID: \"47f5a573-8ada-41c9-b55d-67c511698be5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"
Jul 09 02:35:47.018575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.018200    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e15e0a7-3794-464b-8af7-87ec9972b022-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:47.018575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.018223    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-accelerators-collector-config\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.021095 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.021074    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.021190 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.021141    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e15e0a7-3794-464b-8af7-87ec9972b022-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:47.021270 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.021246    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47f5a573-8ada-41c9-b55d-67c511698be5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-c5872\" (UID: \"47f5a573-8ada-41c9-b55d-67c511698be5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"
Jul 09 02:35:47.021347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.021274    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f5a573-8ada-41c9-b55d-67c511698be5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-c5872\" (UID: \"47f5a573-8ada-41c9-b55d-67c511698be5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"
Jul 09 02:35:47.025692 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.025668    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgt62\" (UniqueName: \"kubernetes.io/projected/47f5a573-8ada-41c9-b55d-67c511698be5-kube-api-access-jgt62\") pod \"openshift-state-metrics-54cf96dfcf-c5872\" (UID: \"47f5a573-8ada-41c9-b55d-67c511698be5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"
Jul 09 02:35:47.025977 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.025960    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57bm2\" (UniqueName: \"kubernetes.io/projected/6e15e0a7-3794-464b-8af7-87ec9972b022-kube-api-access-57bm2\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:47.026432 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.026404    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxtkx\" (UniqueName: \"kubernetes.io/projected/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-kube-api-access-jxtkx\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.156447 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.156409    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"
Jul 09 02:35:47.282558 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.282529    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6gjwz"
Jul 09 02:35:47.522716 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.522627    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e15e0a7-3794-464b-8af7-87ec9972b022-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:47.522956 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.522729    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-tls\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.525516 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.525487    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2c5d0295-77c6-4ea3-9826-a7978bbfb31c-node-exporter-tls\") pod \"node-exporter-kplfz\" (UID: \"2c5d0295-77c6-4ea3-9826-a7978bbfb31c\") " pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.525516 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.525494    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e15e0a7-3794-464b-8af7-87ec9972b022-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-zlnbl\" (UID: \"6e15e0a7-3794-464b-8af7-87ec9972b022\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:47.778028 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.777951    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kplfz"
Jul 09 02:35:47.784135 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:47.784099    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"
Jul 09 02:35:48.539555 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:35:48.539526    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c5d0295_77c6_4ea3_9826_a7978bbfb31c.slice/crio-6a3550c0aae1de4b3907574fd0b5c931c245d29ddef3c794146836be35b9a319 WatchSource:0}: Error finding container 6a3550c0aae1de4b3907574fd0b5c931c245d29ddef3c794146836be35b9a319: Status 404 returned error can't find the container with id 6a3550c0aae1de4b3907574fd0b5c931c245d29ddef3c794146836be35b9a319
Jul 09 02:35:48.682267 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:48.682199    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl"]
Jul 09 02:35:48.689570 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:35:48.689538    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e15e0a7_3794_464b_8af7_87ec9972b022.slice/crio-3303cff376bfbacfe498954d3956cd94ea3a3ac21121898210e62f57c42bd26d WatchSource:0}: Error finding container 3303cff376bfbacfe498954d3956cd94ea3a3ac21121898210e62f57c42bd26d: Status 404 returned error can't find the container with id 3303cff376bfbacfe498954d3956cd94ea3a3ac21121898210e62f57c42bd26d
Jul 09 02:35:48.707045 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:48.706893    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872"]
Jul 09 02:35:48.713710 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:35:48.713200    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47f5a573_8ada_41c9_b55d_67c511698be5.slice/crio-c231f367568a8dcabffc5dcbd2921c02de28db34e3a0536de76791bfed741e9f WatchSource:0}: Error finding container c231f367568a8dcabffc5dcbd2921c02de28db34e3a0536de76791bfed741e9f: Status 404 returned error can't find the container with id c231f367568a8dcabffc5dcbd2921c02de28db34e3a0536de76791bfed741e9f
Jul 09 02:35:49.366595 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.366528    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zc8cn" event={"ID":"9b76c7f8-c7e2-48dd-a392-90409477fb13","Type":"ContainerStarted","Data":"f2e9bdc919b9a174e1959c08d8d2493b8330d55a78c6724b91c8d4c038c35dbd"}
Jul 09 02:35:49.368738 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.368707    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j99cn" event={"ID":"d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa","Type":"ContainerStarted","Data":"40dcbd3b87f2328b8fa44b501883cf2e8e1b4efa49ce9f65ec90b69160fbffd7"}
Jul 09 02:35:49.368864 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.368763    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j99cn" event={"ID":"d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa","Type":"ContainerStarted","Data":"e792ba775eeb6afdddaf58f482fd09a8bcc4a15e82ac6f16c635b5c2735b4e4c"}
Jul 09 02:35:49.368864 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.368804    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-j99cn"
Jul 09 02:35:49.370736 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.370577    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6b29q" event={"ID":"c2415615-0367-4de5-b7f8-afd1945d4b4e","Type":"ContainerStarted","Data":"52c669a1a46525a0d95ec1d9082fc0f58ce47b33133421ede723db83bcacf7ff"}
Jul 09 02:35:49.372043 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.372018    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl" event={"ID":"6e15e0a7-3794-464b-8af7-87ec9972b022","Type":"ContainerStarted","Data":"3303cff376bfbacfe498954d3956cd94ea3a3ac21121898210e62f57c42bd26d"}
Jul 09 02:35:49.373629 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.373600    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kplfz" event={"ID":"2c5d0295-77c6-4ea3-9826-a7978bbfb31c","Type":"ContainerStarted","Data":"6a3550c0aae1de4b3907574fd0b5c931c245d29ddef3c794146836be35b9a319"}
Jul 09 02:35:49.375876 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.375852    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872" event={"ID":"47f5a573-8ada-41c9-b55d-67c511698be5","Type":"ContainerStarted","Data":"aab1bdd8c0658f227bafe65c718caee0c4bc3b679b0650b3a5949765b78ed7cb"}
Jul 09 02:35:49.375962 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.375905    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872" event={"ID":"47f5a573-8ada-41c9-b55d-67c511698be5","Type":"ContainerStarted","Data":"3c8fa0ac75e33c7443452d49c463b43e2025481659bc4c0cdeb970bfeb828160"}
Jul 09 02:35:49.375962 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.375932    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872" event={"ID":"47f5a573-8ada-41c9-b55d-67c511698be5","Type":"ContainerStarted","Data":"c231f367568a8dcabffc5dcbd2921c02de28db34e3a0536de76791bfed741e9f"}
Jul 09 02:35:49.386094 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.386042    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-j99cn" podStartSLOduration=2.265298682 podStartE2EDuration="7.386024326s" totalImagesPullingTime="5.120725644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:35:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:49.384728575 +0000 UTC m=+63.778531992" watchObservedRunningTime="2026-07-09 02:35:49.386024326 +0000 UTC m=+63.779827743"
Jul 09 02:35:49.399199 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.399153    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6b29q" podStartSLOduration=2.269335135 podStartE2EDuration="7.399136677s" totalImagesPullingTime="5.129801542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:35:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:49.398257639 +0000 UTC m=+63.792061054" watchObservedRunningTime="2026-07-09 02:35:49.399136677 +0000 UTC m=+63.792940117"
Jul 09 02:35:49.940129 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.940048    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"]
Jul 09 02:35:49.944085 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.944061    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:49.946672 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.946482    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 09 02:35:49.946672 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.946512    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 09 02:35:49.946672 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.946605    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 09 02:35:49.947011 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.946812    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 09 02:35:49.947011 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.946817    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-zcqgh\""
Jul 09 02:35:49.947011 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.946873    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 09 02:35:49.947224 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.947095    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-792ngte54ohi0\""
Jul 09 02:35:49.956075 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:49.956040    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"]
Jul 09 02:35:50.048391 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.048357    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-thanos-querier-tls\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.048551 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.048465    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.048551 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.048513    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.048551 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.048536    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/542f470c-85ec-4d82-9064-ff81b0b3177e-metrics-client-ca\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.048719 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.048557    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-grpc-tls\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.048719 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.048606    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.048719 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.048647    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.048719 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.048692    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttjj8\" (UniqueName: \"kubernetes.io/projected/542f470c-85ec-4d82-9064-ff81b0b3177e-kube-api-access-ttjj8\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.149937 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.149901    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.150094 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.149953    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.150094 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.149996    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttjj8\" (UniqueName: \"kubernetes.io/projected/542f470c-85ec-4d82-9064-ff81b0b3177e-kube-api-access-ttjj8\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.150094 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.150067    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-thanos-querier-tls\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.150259 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.150120    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.150259 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.150168    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.150259 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.150200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/542f470c-85ec-4d82-9064-ff81b0b3177e-metrics-client-ca\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.150259 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.150225    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-grpc-tls\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.151350 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.151297    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/542f470c-85ec-4d82-9064-ff81b0b3177e-metrics-client-ca\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.153261 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.153228    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-thanos-querier-tls\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.153916 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.153339    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.153916 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.153498    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.153916 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.153869    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.154104 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.153988    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.154155 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.154099    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/542f470c-85ec-4d82-9064-ff81b0b3177e-secret-grpc-tls\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.157513 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.157490    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttjj8\" (UniqueName: \"kubernetes.io/projected/542f470c-85ec-4d82-9064-ff81b0b3177e-kube-api-access-ttjj8\") pod \"thanos-querier-66fd8c76dd-t2grq\" (UID: \"542f470c-85ec-4d82-9064-ff81b0b3177e\") " pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.256639 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.256564    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:35:50.382440 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.382402    2578 generic.go:356] "Generic (PLEG): container finished" podID="2c5d0295-77c6-4ea3-9826-a7978bbfb31c" containerID="000573d84c1ee329e8ab38406b1795383eb83b572a872a7ad2eefc4ee4a6e354" exitCode=0
Jul 09 02:35:50.382591 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.382499    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kplfz" event={"ID":"2c5d0295-77c6-4ea3-9826-a7978bbfb31c","Type":"ContainerDied","Data":"000573d84c1ee329e8ab38406b1795383eb83b572a872a7ad2eefc4ee4a6e354"}
Jul 09 02:35:50.384799 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.384751    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zc8cn" event={"ID":"9b76c7f8-c7e2-48dd-a392-90409477fb13","Type":"ContainerStarted","Data":"3d30636629fd0dec06a4ccd169e4dc9d667663d0cc9094c09fb0fd17929186fe"}
Jul 09 02:35:50.706549 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:50.706382    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"]
Jul 09 02:35:51.150231 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.150200    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"]
Jul 09 02:35:51.155683 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.155657    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.157659 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.157638    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-sj26t\""
Jul 09 02:35:51.157787 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.157692    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 09 02:35:51.157945 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.157921    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ephvhj1790o2c\""
Jul 09 02:35:51.158176 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.158158    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 09 02:35:51.158253 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.158162    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 09 02:35:51.158371 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.158349    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 09 02:35:51.161360 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.161336    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"]
Jul 09 02:35:51.262647 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.262605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a23f1777-3ef1-42e8-9078-ff2a3176fda1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.262822 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.262669    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a23f1777-3ef1-42e8-9078-ff2a3176fda1-secret-metrics-server-client-certs\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.262822 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.262718    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a23f1777-3ef1-42e8-9078-ff2a3176fda1-audit-log\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.262822 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.262758    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a23f1777-3ef1-42e8-9078-ff2a3176fda1-client-ca-bundle\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.262822 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.262796    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a23f1777-3ef1-42e8-9078-ff2a3176fda1-secret-metrics-server-tls\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.263012 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.262844    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a23f1777-3ef1-42e8-9078-ff2a3176fda1-metrics-server-audit-profiles\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.263012 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.262886    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f95v\" (UniqueName: \"kubernetes.io/projected/a23f1777-3ef1-42e8-9078-ff2a3176fda1-kube-api-access-6f95v\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.363880 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.363848    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a23f1777-3ef1-42e8-9078-ff2a3176fda1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.364055 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.363894    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a23f1777-3ef1-42e8-9078-ff2a3176fda1-secret-metrics-server-client-certs\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.364055 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.363947    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a23f1777-3ef1-42e8-9078-ff2a3176fda1-audit-log\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.364055 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.363973    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a23f1777-3ef1-42e8-9078-ff2a3176fda1-client-ca-bundle\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.364055 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.363999    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a23f1777-3ef1-42e8-9078-ff2a3176fda1-secret-metrics-server-tls\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.364055 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.364049    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a23f1777-3ef1-42e8-9078-ff2a3176fda1-metrics-server-audit-profiles\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.364356 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.364086    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f95v\" (UniqueName: \"kubernetes.io/projected/a23f1777-3ef1-42e8-9078-ff2a3176fda1-kube-api-access-6f95v\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.364747 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.364695    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a23f1777-3ef1-42e8-9078-ff2a3176fda1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.365044 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.365024    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a23f1777-3ef1-42e8-9078-ff2a3176fda1-audit-log\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.365494 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.365446    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a23f1777-3ef1-42e8-9078-ff2a3176fda1-metrics-server-audit-profiles\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.367380 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.367356    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a23f1777-3ef1-42e8-9078-ff2a3176fda1-secret-metrics-server-client-certs\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.367703 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.367678    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a23f1777-3ef1-42e8-9078-ff2a3176fda1-secret-metrics-server-tls\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.368094 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.368075    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a23f1777-3ef1-42e8-9078-ff2a3176fda1-client-ca-bundle\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.372081 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.372058    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f95v\" (UniqueName: \"kubernetes.io/projected/a23f1777-3ef1-42e8-9078-ff2a3176fda1-kube-api-access-6f95v\") pod \"metrics-server-7f7b4c4ff5-qfsxt\" (UID: \"a23f1777-3ef1-42e8-9078-ff2a3176fda1\") " pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.392619 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.392585    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq" event={"ID":"542f470c-85ec-4d82-9064-ff81b0b3177e","Type":"ContainerStarted","Data":"8d1bd8613b33f35eb54846e56c7f733f4de74b2c9d1934036e03a031b312b916"}
Jul 09 02:35:51.395507 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.395226    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl" event={"ID":"6e15e0a7-3794-464b-8af7-87ec9972b022","Type":"ContainerStarted","Data":"91267c36f127e228965477d2c5e1dd72a309bbce9c270c6e9fedc08330925a6b"}
Jul 09 02:35:51.395507 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.395281    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl" event={"ID":"6e15e0a7-3794-464b-8af7-87ec9972b022","Type":"ContainerStarted","Data":"8133ac210f85c647f70f14e2f843d5ce906f70a2177fa20798e72af2b1b24e67"}
Jul 09 02:35:51.395507 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.395307    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl" event={"ID":"6e15e0a7-3794-464b-8af7-87ec9972b022","Type":"ContainerStarted","Data":"c3d47a0958a6532f60eda838702a158ffd084da1e951b5484be11d5c75f05a7c"}
Jul 09 02:35:51.398113 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.398054    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kplfz" event={"ID":"2c5d0295-77c6-4ea3-9826-a7978bbfb31c","Type":"ContainerStarted","Data":"0a6f495f3d5383b5a63b37372ff34d51aa4cf7fd39156ed9c2342303d6820d78"}
Jul 09 02:35:51.398113 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.398101    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kplfz" event={"ID":"2c5d0295-77c6-4ea3-9826-a7978bbfb31c","Type":"ContainerStarted","Data":"ada737f484cad95b68f655e3e5b12d331e9d7d35d2b02eb6924fe8eb416b2a20"}
Jul 09 02:35:51.400682 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.400623    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872" event={"ID":"47f5a573-8ada-41c9-b55d-67c511698be5","Type":"ContainerStarted","Data":"e974a5d49a15e588c3c548536bb0fdfa70fe47d513dc84d1ad0d0dde35ec4f6b"}
Jul 09 02:35:51.412288 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.412236    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zlnbl" podStartSLOduration=3.565252731 podStartE2EDuration="5.412217745s" totalImagesPullingTime="1.846965014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:51.410951733 +0000 UTC m=+65.804755147" watchObservedRunningTime="2026-07-09 02:35:51.412217745 +0000 UTC m=+65.806021159"
Jul 09 02:35:51.434310 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.434258    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-c5872" podStartSLOduration=3.751338154 podStartE2EDuration="5.434240783s" totalImagesPullingTime="1.682902629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:51.433515493 +0000 UTC m=+65.827318907" watchObservedRunningTime="2026-07-09 02:35:51.434240783 +0000 UTC m=+65.828044188"
Jul 09 02:35:51.450582 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.450502    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kplfz" podStartSLOduration=4.335735173 podStartE2EDuration="5.450485641s" totalImagesPullingTime="1.051942244s" totalInitContainerRuntime="62.808224ms" isStatefulPod=true podCreationTimestamp="2026-07-09 02:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:51.449987751 +0000 UTC m=+65.843791164" watchObservedRunningTime="2026-07-09 02:35:51.450485641 +0000 UTC m=+65.844289031"
Jul 09 02:35:51.467578 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.467552    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:35:51.768556 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.768451    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs\") pod \"network-metrics-daemon-jt6w4\" (UID: \"29ffb437-ae69-4765-85d3-5f1ebe2c0346\") " pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:51.770607 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.770563    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 02:35:51.781726 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.781700    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ffb437-ae69-4765-85d3-5f1ebe2c0346-metrics-certs\") pod \"network-metrics-daemon-jt6w4\" (UID: \"29ffb437-ae69-4765-85d3-5f1ebe2c0346\") " pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:51.869748 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.869703    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret\") pod \"global-pull-secret-syncer-t6lpr\" (UID: \"759cd68b-755a-47d5-bb83-7a9028adbc86\") " pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:51.869907 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.869793    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dwnz\" (UniqueName: \"kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz\") pod \"network-check-target-ppkl8\" (UID: \"f6f3333d-e691-4ae1-8ee6-55f30813e35d\") " pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:51.871903 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.871878    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 02:35:51.872036 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.871879    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 02:35:51.882634 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.882458    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 02:35:51.883790 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.883735    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/759cd68b-755a-47d5-bb83-7a9028adbc86-original-pull-secret\") pod \"global-pull-secret-syncer-t6lpr\" (UID: \"759cd68b-755a-47d5-bb83-7a9028adbc86\") " pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:51.894121 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.894086    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dwnz\" (UniqueName: \"kubernetes.io/projected/f6f3333d-e691-4ae1-8ee6-55f30813e35d-kube-api-access-7dwnz\") pod \"network-check-target-ppkl8\" (UID: \"f6f3333d-e691-4ae1-8ee6-55f30813e35d\") " pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:51.931131 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.931096    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9ptb2\""
Jul 09 02:35:51.938829 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.938730    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t6lpr"
Jul 09 02:35:51.939855 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.939778    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:51.947594 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.947573    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w8vs7\""
Jul 09 02:35:51.956149 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:51.956127    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jt6w4"
Jul 09 02:35:52.066210 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.066139    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"]
Jul 09 02:35:52.069777 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.069759    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.071915 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.071765    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 09 02:35:52.071915 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.071779    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 09 02:35:52.071915 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.071799    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 09 02:35:52.071915 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.071765    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 09 02:35:52.071915 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.071867    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-x88d2\""
Jul 09 02:35:52.072235 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.072151    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 09 02:35:52.077490 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.077464    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 09 02:35:52.082859 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.082839    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"]
Jul 09 02:35:52.171890 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.171854    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25533b64-b04c-4912-96ec-caa77912b43e-serving-certs-ca-bundle\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.172375 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.171909    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25533b64-b04c-4912-96ec-caa77912b43e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.172375 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.171995    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz9px\" (UniqueName: \"kubernetes.io/projected/25533b64-b04c-4912-96ec-caa77912b43e-kube-api-access-zz9px\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.172375 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.172076    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/25533b64-b04c-4912-96ec-caa77912b43e-secret-telemeter-client\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.172375 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.172114    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25533b64-b04c-4912-96ec-caa77912b43e-metrics-client-ca\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.172375 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.172158    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/25533b64-b04c-4912-96ec-caa77912b43e-telemeter-client-tls\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.172375 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.172232    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25533b64-b04c-4912-96ec-caa77912b43e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.172375 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.172298    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/25533b64-b04c-4912-96ec-caa77912b43e-federate-client-tls\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.273647 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.273605    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/25533b64-b04c-4912-96ec-caa77912b43e-federate-client-tls\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.273816 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.273665    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25533b64-b04c-4912-96ec-caa77912b43e-serving-certs-ca-bundle\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.273816 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.273693    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25533b64-b04c-4912-96ec-caa77912b43e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.273816 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.273728    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz9px\" (UniqueName: \"kubernetes.io/projected/25533b64-b04c-4912-96ec-caa77912b43e-kube-api-access-zz9px\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.273816 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.273766    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/25533b64-b04c-4912-96ec-caa77912b43e-secret-telemeter-client\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.273816 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.273788    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25533b64-b04c-4912-96ec-caa77912b43e-metrics-client-ca\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.274068 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.273824    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/25533b64-b04c-4912-96ec-caa77912b43e-telemeter-client-tls\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.274068 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.273862    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25533b64-b04c-4912-96ec-caa77912b43e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.274513 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.274465    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25533b64-b04c-4912-96ec-caa77912b43e-serving-certs-ca-bundle\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.274808 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.274785    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25533b64-b04c-4912-96ec-caa77912b43e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.274889 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.274789    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25533b64-b04c-4912-96ec-caa77912b43e-metrics-client-ca\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.277001 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.276974    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/25533b64-b04c-4912-96ec-caa77912b43e-federate-client-tls\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.277106 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.277029    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/25533b64-b04c-4912-96ec-caa77912b43e-secret-telemeter-client\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.277106 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.277070    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25533b64-b04c-4912-96ec-caa77912b43e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.277512 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.277473    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/25533b64-b04c-4912-96ec-caa77912b43e-telemeter-client-tls\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.281189 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.281169    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz9px\" (UniqueName: \"kubernetes.io/projected/25533b64-b04c-4912-96ec-caa77912b43e-kube-api-access-zz9px\") pod \"telemeter-client-5c7d6df564-rhbv6\" (UID: \"25533b64-b04c-4912-96ec-caa77912b43e\") " pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:52.380559 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:52.380526    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"
Jul 09 02:35:53.071578 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.071537    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 02:35:53.075818 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.075785    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.078334 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.078167    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 09 02:35:53.078334 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.078205    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 09 02:35:53.078334 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.078279    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 09 02:35:53.078334 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.078294    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 09 02:35:53.078661 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.078572    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 09 02:35:53.078661 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.078592    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 09 02:35:53.078661 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.078644    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 09 02:35:53.078912 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.078893    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 09 02:35:53.079151 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.079021    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 09 02:35:53.079151 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.079105    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-45rc09dnngj0q\""
Jul 09 02:35:53.079342 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.079231    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 09 02:35:53.079418 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.079348    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 09 02:35:53.079418 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.079374    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5jfhb\""
Jul 09 02:35:53.079418 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.079413    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 09 02:35:53.080007 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.079836    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 09 02:35:53.081180 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.081161    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 09 02:35:53.093439 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.093417    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 02:35:53.181540 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181509    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.181900 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181547    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78bqw\" (UniqueName: \"kubernetes.io/projected/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-kube-api-access-78bqw\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.181900 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181571    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.181900 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181594    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-web-config\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.181900 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181625    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.181900 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181649    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.181900 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181677    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.181900 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181709    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.181900 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181742    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.181900 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181774    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.181900 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181814    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.181900 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181845    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.181900 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181873    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.181900 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181901    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-config-out\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.182733 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181927    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.182733 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181967    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.182733 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.181995    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.182733 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.182016    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.182733 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.182049    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-config\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.182733 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.182073    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.284843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283082    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.284843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283124    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.284843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.284843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283257    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-config\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.284843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283279    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.284843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283309    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.284843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283349    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78bqw\" (UniqueName: \"kubernetes.io/projected/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-kube-api-access-78bqw\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.284843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283374    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.284843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283417    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-web-config\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.284843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283451    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.284843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283476    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.284843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283504    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.284843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283540    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.284843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283580    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.285594 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283616    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.285594 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283659    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.285594 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283694    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.285594 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283732    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.285594 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283763    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-config-out\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.285594 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.283795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.292146 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.286931    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.292146 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.288417    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.292146 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.289230    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.292146 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.289385    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.292146 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.290289    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.292146 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.291688    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.293118 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.293052    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.293681 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.293344    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.298076 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.297245    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-web-config\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.298076 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.297457    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-config-out\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.298076 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.297582    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.298076 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.298015    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.300027 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.298679    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.300027 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.298898    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.300027 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.299458    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.300027 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.299915    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.301865 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.301138    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.301865 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.301672    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-config\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.301865 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.301752    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.304677 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.304656    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78bqw\" (UniqueName: \"kubernetes.io/projected/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-kube-api-access-78bqw\") pod \"prometheus-k8s-0\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.360000 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.359359    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5c7d6df564-rhbv6"]
Jul 09 02:35:53.363202 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:35:53.363165    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25533b64_b04c_4912_96ec_caa77912b43e.slice/crio-4444b04bd4f8eda40e9d8397a9bae8f143bf56e5f3180b6970d0e53e96620ad7 WatchSource:0}: Error finding container 4444b04bd4f8eda40e9d8397a9bae8f143bf56e5f3180b6970d0e53e96620ad7: Status 404 returned error can't find the container with id 4444b04bd4f8eda40e9d8397a9bae8f143bf56e5f3180b6970d0e53e96620ad7
Jul 09 02:35:53.388675 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.388651    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:35:53.413571 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.413524    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jt6w4"]
Jul 09 02:35:53.418033 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:35:53.417993    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ffb437_ae69_4765_85d3_5f1ebe2c0346.slice/crio-38f3c5e068183ddfd0fb903955d76801033bf8025bb2b15a04f2e74f6cb4dda0 WatchSource:0}: Error finding container 38f3c5e068183ddfd0fb903955d76801033bf8025bb2b15a04f2e74f6cb4dda0: Status 404 returned error can't find the container with id 38f3c5e068183ddfd0fb903955d76801033bf8025bb2b15a04f2e74f6cb4dda0
Jul 09 02:35:53.420649 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.420580    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zc8cn" event={"ID":"9b76c7f8-c7e2-48dd-a392-90409477fb13","Type":"ContainerStarted","Data":"d41114751e4d809bafbcaf2e4b1618cc8a58a273e3d95cacdf0911dda9c31d5d"}
Jul 09 02:35:53.424098 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.424073    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq" event={"ID":"542f470c-85ec-4d82-9064-ff81b0b3177e","Type":"ContainerStarted","Data":"d35faa2871292aa3544ce498ba27b6ee0a9592fd79cb3d731cd16c6e47d6cc59"}
Jul 09 02:35:53.424178 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.424134    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq" event={"ID":"542f470c-85ec-4d82-9064-ff81b0b3177e","Type":"ContainerStarted","Data":"c1b48b60a4bde90bf6315317660c666211eee51bda79afbce84e4677b95a05ec"}
Jul 09 02:35:53.425356 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.425334    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6" event={"ID":"25533b64-b04c-4912-96ec-caa77912b43e","Type":"ContainerStarted","Data":"4444b04bd4f8eda40e9d8397a9bae8f143bf56e5f3180b6970d0e53e96620ad7"}
Jul 09 02:35:53.427931 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.427879    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t6lpr"]
Jul 09 02:35:53.431859 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:35:53.431836    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod759cd68b_755a_47d5_bb83_7a9028adbc86.slice/crio-7a4773c11101ec034bf141f2a81f1a46d2aa6cef601a96a615c9975100ab5347 WatchSource:0}: Error finding container 7a4773c11101ec034bf141f2a81f1a46d2aa6cef601a96a615c9975100ab5347: Status 404 returned error can't find the container with id 7a4773c11101ec034bf141f2a81f1a46d2aa6cef601a96a615c9975100ab5347
Jul 09 02:35:53.443353 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.442899    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zc8cn" podStartSLOduration=1.83246004 podStartE2EDuration="11.442886452s" totalImagesPullingTime="9.610426412s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 02:35:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:53.442373925 +0000 UTC m=+67.836177338" watchObservedRunningTime="2026-07-09 02:35:53.442886452 +0000 UTC m=+67.836689902"
Jul 09 02:35:53.541659 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.541624    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 02:35:53.544482 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:35:53.544444    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f35f9e9_3eb7_4bf4_b9e4_c67ffdc9045c.slice/crio-2e90d951445d2adfaff20c24428e18cf7b20d688780595b02b3c997de110a468 WatchSource:0}: Error finding container 2e90d951445d2adfaff20c24428e18cf7b20d688780595b02b3c997de110a468: Status 404 returned error can't find the container with id 2e90d951445d2adfaff20c24428e18cf7b20d688780595b02b3c997de110a468
Jul 09 02:35:53.576213 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.576141    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ppkl8"]
Jul 09 02:35:53.579035 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:35:53.579008    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f3333d_e691_4ae1_8ee6_55f30813e35d.slice/crio-123ce0a2cc156b3e2bf43bfbd26083eb3c1f77b1ceb9c462b7ca4fd79cbe6867 WatchSource:0}: Error finding container 123ce0a2cc156b3e2bf43bfbd26083eb3c1f77b1ceb9c462b7ca4fd79cbe6867: Status 404 returned error can't find the container with id 123ce0a2cc156b3e2bf43bfbd26083eb3c1f77b1ceb9c462b7ca4fd79cbe6867
Jul 09 02:35:53.579819 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:53.579747    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"]
Jul 09 02:35:53.582523 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:35:53.582500    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda23f1777_3ef1_42e8_9078_ff2a3176fda1.slice/crio-c74477734b70cad2eff85dafd02d47fee923b8dae1dcf37802fc84be8247880c WatchSource:0}: Error finding container c74477734b70cad2eff85dafd02d47fee923b8dae1dcf37802fc84be8247880c: Status 404 returned error can't find the container with id c74477734b70cad2eff85dafd02d47fee923b8dae1dcf37802fc84be8247880c
Jul 09 02:35:54.433709 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:54.433669    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c","Type":"ContainerStarted","Data":"2e90d951445d2adfaff20c24428e18cf7b20d688780595b02b3c997de110a468"}
Jul 09 02:35:54.437455 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:54.437416    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jt6w4" event={"ID":"29ffb437-ae69-4765-85d3-5f1ebe2c0346","Type":"ContainerStarted","Data":"38f3c5e068183ddfd0fb903955d76801033bf8025bb2b15a04f2e74f6cb4dda0"}
Jul 09 02:35:54.439624 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:54.439593    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt" event={"ID":"a23f1777-3ef1-42e8-9078-ff2a3176fda1","Type":"ContainerStarted","Data":"c74477734b70cad2eff85dafd02d47fee923b8dae1dcf37802fc84be8247880c"}
Jul 09 02:35:54.453039 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:54.453011    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq" event={"ID":"542f470c-85ec-4d82-9064-ff81b0b3177e","Type":"ContainerStarted","Data":"5f4dc361cb6bc9edeecbeb9f57d1dfb4c5b3160422d3cb292a7cb1de2f85f76b"}
Jul 09 02:35:54.463197 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:54.463138    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ppkl8" event={"ID":"f6f3333d-e691-4ae1-8ee6-55f30813e35d","Type":"ContainerStarted","Data":"123ce0a2cc156b3e2bf43bfbd26083eb3c1f77b1ceb9c462b7ca4fd79cbe6867"}
Jul 09 02:35:54.465681 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:54.465614    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t6lpr" event={"ID":"759cd68b-755a-47d5-bb83-7a9028adbc86","Type":"ContainerStarted","Data":"7a4773c11101ec034bf141f2a81f1a46d2aa6cef601a96a615c9975100ab5347"}
Jul 09 02:35:57.483010 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:57.482969    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq" event={"ID":"542f470c-85ec-4d82-9064-ff81b0b3177e","Type":"ContainerStarted","Data":"85e55d6098441b61ec706275e8acd4c47372d587bbdf12d76389028cfafb2150"}
Jul 09 02:35:57.483470 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:57.483100    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq" event={"ID":"542f470c-85ec-4d82-9064-ff81b0b3177e","Type":"ContainerStarted","Data":"cb3bdbe826eb3699687220c460fab9db0450dd556a87e3df5e7e4c3579e3aea4"}
Jul 09 02:35:57.485115 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:57.485080    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6" event={"ID":"25533b64-b04c-4912-96ec-caa77912b43e","Type":"ContainerStarted","Data":"020e6308b07ad20bfbee632151e2d5153c31767ca9de71456f79da6953b8723b"}
Jul 09 02:35:57.485255 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:57.485143    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6" event={"ID":"25533b64-b04c-4912-96ec-caa77912b43e","Type":"ContainerStarted","Data":"4f0d44b926fd5312d3bbfe2e5bfebf994c25f6af80177414d13072c90f78a9e9"}
Jul 09 02:35:57.486904 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:57.486875    2578 generic.go:356] "Generic (PLEG): container finished" podID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerID="736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08" exitCode=0
Jul 09 02:35:57.487039 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:57.486961    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c","Type":"ContainerDied","Data":"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08"}
Jul 09 02:35:57.489005 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:57.488977    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jt6w4" event={"ID":"29ffb437-ae69-4765-85d3-5f1ebe2c0346","Type":"ContainerStarted","Data":"dc10c9d85da3905f191764431a1683af9dabf530a46352ff7a04e3a52d7e9cb6"}
Jul 09 02:35:57.489109 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:57.489029    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jt6w4" event={"ID":"29ffb437-ae69-4765-85d3-5f1ebe2c0346","Type":"ContainerStarted","Data":"c0119e09c0eea2ac0fc79829e04e0c39766de31ce347fd868d74b5b55f3f0d34"}
Jul 09 02:35:57.490916 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:57.490889    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt" event={"ID":"a23f1777-3ef1-42e8-9078-ff2a3176fda1","Type":"ContainerStarted","Data":"3fbe30f063bafec235978e26e6092df0c0fd6da7bb69ad0751bbfad47f1c340b"}
Jul 09 02:35:57.527581 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:57.527533    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jt6w4" podStartSLOduration=68.432501267 podStartE2EDuration="1m11.527515568s" totalImagesPullingTime="3.095014301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:34:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:57.52604947 +0000 UTC m=+71.919852886" watchObservedRunningTime="2026-07-09 02:35:57.527515568 +0000 UTC m=+71.921318982"
Jul 09 02:35:57.542207 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:57.542162    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt" podStartSLOduration=3.609180656 podStartE2EDuration="6.542147669s" totalImagesPullingTime="2.932967013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:35:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:57.540937146 +0000 UTC m=+71.934740561" watchObservedRunningTime="2026-07-09 02:35:57.542147669 +0000 UTC m=+71.935951082"
Jul 09 02:35:59.387609 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:59.387556    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-j99cn"
Jul 09 02:35:59.510005 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:59.509964    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:35:59.530754 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:59.530705    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6" podStartSLOduration=4.380957977 podStartE2EDuration="7.53068543s" totalImagesPullingTime="3.149727453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:35:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:59.527760766 +0000 UTC m=+73.921564180" watchObservedRunningTime="2026-07-09 02:35:59.53068543 +0000 UTC m=+73.924488842"
Jul 09 02:35:59.543305 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:35:59.543245    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ppkl8" podStartSLOduration=67.730974655 podStartE2EDuration="1m13.543220765s" totalImagesPullingTime="5.81224611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:34:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:35:59.542477022 +0000 UTC m=+73.936280436" watchObservedRunningTime="2026-07-09 02:35:59.543220765 +0000 UTC m=+73.937024179"
Jul 09 02:36:00.518902 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:00.518862    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq" event={"ID":"542f470c-85ec-4d82-9064-ff81b0b3177e","Type":"ContainerStarted","Data":"3a02d0b8da77add0b0e01e1a11379895e1b64da57ca0250b2befbc8f3d3a59fb"}
Jul 09 02:36:00.519429 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:00.518990    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:36:00.521562 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:00.521527    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c7d6df564-rhbv6" event={"ID":"25533b64-b04c-4912-96ec-caa77912b43e","Type":"ContainerStarted","Data":"893299f0ba3c777f8b9cecb46d8e0e44d276c3843de5d52109e78f749891e360"}
Jul 09 02:36:00.524125 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:00.524101    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ppkl8" event={"ID":"f6f3333d-e691-4ae1-8ee6-55f30813e35d","Type":"ContainerStarted","Data":"ca4421351b925ce1b5fe47b6eea812b5818e0bc1a882b9619ba1124cd599e2ad"}
Jul 09 02:36:00.525844 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:00.525823    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t6lpr" event={"ID":"759cd68b-755a-47d5-bb83-7a9028adbc86","Type":"ContainerStarted","Data":"53770071f39f7ce57506def83916963cc67eb501959bee88e3ae80bf6083c493"}
Jul 09 02:36:00.527092 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:00.527075    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq"
Jul 09 02:36:00.541082 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:00.541002    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-66fd8c76dd-t2grq" podStartSLOduration=6.037750876 podStartE2EDuration="11.540983791s" totalImagesPullingTime="5.503232915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:35:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:36:00.53957988 +0000 UTC m=+74.933383295" watchObservedRunningTime="2026-07-09 02:36:00.540983791 +0000 UTC m=+74.934787205"
Jul 09 02:36:00.558338 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:00.558283    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-t6lpr" podStartSLOduration=67.599046976 podStartE2EDuration="1m13.558270317s" totalImagesPullingTime="5.959223341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 02:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:36:00.55667447 +0000 UTC m=+74.950477882" watchObservedRunningTime="2026-07-09 02:36:00.558270317 +0000 UTC m=+74.952073730"
Jul 09 02:36:03.543718 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:03.543679    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c","Type":"ContainerStarted","Data":"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3"}
Jul 09 02:36:03.544250 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:03.543757    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c","Type":"ContainerStarted","Data":"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800"}
Jul 09 02:36:03.544250 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:03.543796    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c","Type":"ContainerStarted","Data":"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82"}
Jul 09 02:36:03.544250 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:03.543838    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c","Type":"ContainerStarted","Data":"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92"}
Jul 09 02:36:03.544250 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:03.543879    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c","Type":"ContainerStarted","Data":"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a"}
Jul 09 02:36:03.544250 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:03.543915    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c","Type":"ContainerStarted","Data":"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91"}
Jul 09 02:36:03.573284 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:03.573234    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.376852908 podStartE2EDuration="10.57321851s" totalImagesPullingTime="8.155483812s" totalInitContainerRuntime="40.88179ms" isStatefulPod=false podCreationTimestamp="2026-07-09 02:35:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:36:03.572902856 +0000 UTC m=+77.966706271" watchObservedRunningTime="2026-07-09 02:36:03.57321851 +0000 UTC m=+77.967021914"
Jul 09 02:36:08.389586 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:08.389476    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:36:11.468356 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:11.468324    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:36:11.468769 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:11.468402    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:36:21.233374 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:21.233337    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6b29q_c2415615-0367-4de5-b7f8-afd1945d4b4e/serve-healthcheck-canary/0.log"
Jul 09 02:36:31.474234 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:31.474205    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:36:31.478236 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:31.478214    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7f7b4c4ff5-qfsxt"
Jul 09 02:36:31.533533 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:31.533505    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ppkl8"
Jul 09 02:36:53.388988 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:53.388953    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:36:53.407151 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:53.407127    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:36:53.742757 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:36:53.742689    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.430546 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.430511    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 02:37:11.430933 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.430810    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="prometheus" containerID="cri-o://288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91" gracePeriod=600
Jul 09 02:37:11.430933 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.430874    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3" gracePeriod=600
Jul 09 02:37:11.431049 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.430912    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="kube-rbac-proxy-web" containerID="cri-o://e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82" gracePeriod=600
Jul 09 02:37:11.431049 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.430920    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="config-reloader" containerID="cri-o://0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a" gracePeriod=600
Jul 09 02:37:11.431049 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.430886    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="thanos-sidecar" containerID="cri-o://5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92" gracePeriod=600
Jul 09 02:37:11.431049 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.430831    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="kube-rbac-proxy" containerID="cri-o://4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800" gracePeriod=600
Jul 09 02:37:11.677410 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.677386    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.756898 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.756814    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-trusted-ca-bundle\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.756898 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.756854    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-tls\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.756898 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.756879    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-metrics-client-certs\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757155 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.756904    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-config\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757155 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.756927    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78bqw\" (UniqueName: \"kubernetes.io/projected/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-kube-api-access-78bqw\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757155 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.756956    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757155 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.756982    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-config-out\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757155 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.756999    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-thanos-prometheus-http-client-file\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757155 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.757036    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-0\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757155 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.757066    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-tls-assets\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757155 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.757094    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-2\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757155 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.757118    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-kube-rbac-proxy\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757155 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.757148    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757668 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.757176    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-metrics-client-ca\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757668 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.757202    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-kubelet-serving-ca-bundle\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757668 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.757241    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-web-config\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757668 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.757266    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-db\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757668 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.757274    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 02:37:11.757668 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.757301    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-serving-certs-ca-bundle\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757668 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.757359    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-1\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757668 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.757383    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-grpc-tls\") pod \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\" (UID: \"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c\") "
Jul 09 02:37:11.757668 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.757650    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.758706 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.758674    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 02:37:11.759149 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.759118    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 02:37:11.760536 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.760440    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 02:37:11.760630 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.760584    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 02:37:11.761176 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.761139    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 02:37:11.761307 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.761231    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 02:37:11.761570 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.761333    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 02:37:11.761570 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.761503    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 02:37:11.761793 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.761771    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-config-out" (OuterVolumeSpecName: "config-out") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 02:37:11.761889 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.761866    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 02:37:11.761955 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.761786    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 02:37:11.761955 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.761819    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-kube-api-access-78bqw" (OuterVolumeSpecName: "kube-api-access-78bqw") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "kube-api-access-78bqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 02:37:11.762061 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.761967    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 02:37:11.762061 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.762044    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-config" (OuterVolumeSpecName: "config") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 02:37:11.762254 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.762231    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 02:37:11.762646 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.762620    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 02:37:11.762742 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.762662    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 02:37:11.762974 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.762957    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 02:37:11.770787 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.770768    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-web-config" (OuterVolumeSpecName: "web-config") pod "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" (UID: "1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 02:37:11.792177 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.792154    2578 generic.go:356] "Generic (PLEG): container finished" podID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerID="bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3" exitCode=0
Jul 09 02:37:11.792177 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.792174    2578 generic.go:356] "Generic (PLEG): container finished" podID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerID="4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800" exitCode=0
Jul 09 02:37:11.792177 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.792180    2578 generic.go:356] "Generic (PLEG): container finished" podID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerID="e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82" exitCode=0
Jul 09 02:37:11.792350 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.792185    2578 generic.go:356] "Generic (PLEG): container finished" podID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerID="5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92" exitCode=0
Jul 09 02:37:11.792350 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.792190    2578 generic.go:356] "Generic (PLEG): container finished" podID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerID="0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a" exitCode=0
Jul 09 02:37:11.792350 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.792195    2578 generic.go:356] "Generic (PLEG): container finished" podID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerID="288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91" exitCode=0
Jul 09 02:37:11.792350 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.792235    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.792350 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.792235    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c","Type":"ContainerDied","Data":"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3"}
Jul 09 02:37:11.792350 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.792307    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c","Type":"ContainerDied","Data":"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800"}
Jul 09 02:37:11.792525 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.792352    2578 scope.go:117] "RemoveContainer" containerID="bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3"
Jul 09 02:37:11.792525 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.792376    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c","Type":"ContainerDied","Data":"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82"}
Jul 09 02:37:11.792525 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.792415    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c","Type":"ContainerDied","Data":"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92"}
Jul 09 02:37:11.792525 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.792456    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c","Type":"ContainerDied","Data":"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a"}
Jul 09 02:37:11.792525 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.792497    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c","Type":"ContainerDied","Data":"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91"}
Jul 09 02:37:11.792729 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.792530    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c","Type":"ContainerDied","Data":"2e90d951445d2adfaff20c24428e18cf7b20d688780595b02b3c997de110a468"}
Jul 09 02:37:11.799928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.799909    2578 scope.go:117] "RemoveContainer" containerID="4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800"
Jul 09 02:37:11.806793 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.806773    2578 scope.go:117] "RemoveContainer" containerID="e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82"
Jul 09 02:37:11.813035 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.813017    2578 scope.go:117] "RemoveContainer" containerID="5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92"
Jul 09 02:37:11.815967 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.815948    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 02:37:11.820161 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.820136    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 02:37:11.820501 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.820489    2578 scope.go:117] "RemoveContainer" containerID="0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a"
Jul 09 02:37:11.826897 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.826884    2578 scope.go:117] "RemoveContainer" containerID="288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91"
Jul 09 02:37:11.833641 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.833626    2578 scope.go:117] "RemoveContainer" containerID="736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08"
Jul 09 02:37:11.840045 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.840028    2578 scope.go:117] "RemoveContainer" containerID="bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3"
Jul 09 02:37:11.840324 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:37:11.840293    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3\": container with ID starting with bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3 not found: ID does not exist" containerID="bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3"
Jul 09 02:37:11.840384 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.840345    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3"} err="failed to get container status \"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3\": rpc error: code = NotFound desc = could not find container \"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3\": container with ID starting with bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3 not found: ID does not exist"
Jul 09 02:37:11.840436 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.840387    2578 scope.go:117] "RemoveContainer" containerID="4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800"
Jul 09 02:37:11.840640 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:37:11.840623    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800\": container with ID starting with 4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800 not found: ID does not exist" containerID="4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800"
Jul 09 02:37:11.840679 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.840646    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800"} err="failed to get container status \"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800\": rpc error: code = NotFound desc = could not find container \"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800\": container with ID starting with 4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800 not found: ID does not exist"
Jul 09 02:37:11.840679 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.840668    2578 scope.go:117] "RemoveContainer" containerID="e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82"
Jul 09 02:37:11.840886 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:37:11.840869    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82\": container with ID starting with e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82 not found: ID does not exist" containerID="e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82"
Jul 09 02:37:11.840964 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.840893    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82"} err="failed to get container status \"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82\": rpc error: code = NotFound desc = could not find container \"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82\": container with ID starting with e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82 not found: ID does not exist"
Jul 09 02:37:11.840964 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.840914    2578 scope.go:117] "RemoveContainer" containerID="5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92"
Jul 09 02:37:11.841146 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:37:11.841127    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92\": container with ID starting with 5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92 not found: ID does not exist" containerID="5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92"
Jul 09 02:37:11.841183 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.841153    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92"} err="failed to get container status \"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92\": rpc error: code = NotFound desc = could not find container \"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92\": container with ID starting with 5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92 not found: ID does not exist"
Jul 09 02:37:11.841183 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.841166    2578 scope.go:117] "RemoveContainer" containerID="0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a"
Jul 09 02:37:11.841393 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:37:11.841371    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a\": container with ID starting with 0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a not found: ID does not exist" containerID="0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a"
Jul 09 02:37:11.841484 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.841394    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a"} err="failed to get container status \"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a\": rpc error: code = NotFound desc = could not find container \"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a\": container with ID starting with 0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a not found: ID does not exist"
Jul 09 02:37:11.841484 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.841408    2578 scope.go:117] "RemoveContainer" containerID="288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91"
Jul 09 02:37:11.841637 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:37:11.841619    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91\": container with ID starting with 288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91 not found: ID does not exist" containerID="288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91"
Jul 09 02:37:11.841677 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.841642    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91"} err="failed to get container status \"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91\": rpc error: code = NotFound desc = could not find container \"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91\": container with ID starting with 288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91 not found: ID does not exist"
Jul 09 02:37:11.841677 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.841658    2578 scope.go:117] "RemoveContainer" containerID="736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08"
Jul 09 02:37:11.841875 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:37:11.841858    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08\": container with ID starting with 736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08 not found: ID does not exist" containerID="736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08"
Jul 09 02:37:11.841923 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.841879    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08"} err="failed to get container status \"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08\": rpc error: code = NotFound desc = could not find container \"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08\": container with ID starting with 736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08 not found: ID does not exist"
Jul 09 02:37:11.841923 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.841891    2578 scope.go:117] "RemoveContainer" containerID="bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3"
Jul 09 02:37:11.842108 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.842091    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3"} err="failed to get container status \"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3\": rpc error: code = NotFound desc = could not find container \"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3\": container with ID starting with bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3 not found: ID does not exist"
Jul 09 02:37:11.842146 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.842109    2578 scope.go:117] "RemoveContainer" containerID="4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800"
Jul 09 02:37:11.842297 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.842281    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800"} err="failed to get container status \"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800\": rpc error: code = NotFound desc = could not find container \"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800\": container with ID starting with 4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800 not found: ID does not exist"
Jul 09 02:37:11.842404 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.842296    2578 scope.go:117] "RemoveContainer" containerID="e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82"
Jul 09 02:37:11.842594 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.842577    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82"} err="failed to get container status \"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82\": rpc error: code = NotFound desc = could not find container \"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82\": container with ID starting with e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82 not found: ID does not exist"
Jul 09 02:37:11.842640 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.842594    2578 scope.go:117] "RemoveContainer" containerID="5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92"
Jul 09 02:37:11.842792 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.842775    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92"} err="failed to get container status \"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92\": rpc error: code = NotFound desc = could not find container \"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92\": container with ID starting with 5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92 not found: ID does not exist"
Jul 09 02:37:11.842831 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.842794    2578 scope.go:117] "RemoveContainer" containerID="0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a"
Jul 09 02:37:11.842989 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.842973    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a"} err="failed to get container status \"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a\": rpc error: code = NotFound desc = could not find container \"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a\": container with ID starting with 0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a not found: ID does not exist"
Jul 09 02:37:11.843039 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.842989    2578 scope.go:117] "RemoveContainer" containerID="288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91"
Jul 09 02:37:11.843198 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.843181    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91"} err="failed to get container status \"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91\": rpc error: code = NotFound desc = could not find container \"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91\": container with ID starting with 288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91 not found: ID does not exist"
Jul 09 02:37:11.843241 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.843200    2578 scope.go:117] "RemoveContainer" containerID="736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08"
Jul 09 02:37:11.843428 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.843410    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08"} err="failed to get container status \"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08\": rpc error: code = NotFound desc = could not find container \"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08\": container with ID starting with 736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08 not found: ID does not exist"
Jul 09 02:37:11.843485 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.843428    2578 scope.go:117] "RemoveContainer" containerID="bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3"
Jul 09 02:37:11.843631 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.843614    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3"} err="failed to get container status \"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3\": rpc error: code = NotFound desc = could not find container \"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3\": container with ID starting with bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3 not found: ID does not exist"
Jul 09 02:37:11.843672 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.843632    2578 scope.go:117] "RemoveContainer" containerID="4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800"
Jul 09 02:37:11.843823 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.843807    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800"} err="failed to get container status \"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800\": rpc error: code = NotFound desc = could not find container \"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800\": container with ID starting with 4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800 not found: ID does not exist"
Jul 09 02:37:11.843873 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.843825    2578 scope.go:117] "RemoveContainer" containerID="e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82"
Jul 09 02:37:11.844019 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.844004    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82"} err="failed to get container status \"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82\": rpc error: code = NotFound desc = could not find container \"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82\": container with ID starting with e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82 not found: ID does not exist"
Jul 09 02:37:11.844055 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.844019    2578 scope.go:117] "RemoveContainer" containerID="5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92"
Jul 09 02:37:11.844195 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.844182    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92"} err="failed to get container status \"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92\": rpc error: code = NotFound desc = could not find container \"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92\": container with ID starting with 5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92 not found: ID does not exist"
Jul 09 02:37:11.844234 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.844194    2578 scope.go:117] "RemoveContainer" containerID="0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a"
Jul 09 02:37:11.844413 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.844395    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a"} err="failed to get container status \"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a\": rpc error: code = NotFound desc = could not find container \"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a\": container with ID starting with 0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a not found: ID does not exist"
Jul 09 02:37:11.844454 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.844413    2578 scope.go:117] "RemoveContainer" containerID="288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91"
Jul 09 02:37:11.844655 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.844609    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91"} err="failed to get container status \"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91\": rpc error: code = NotFound desc = could not find container \"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91\": container with ID starting with 288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91 not found: ID does not exist"
Jul 09 02:37:11.844655 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.844636    2578 scope.go:117] "RemoveContainer" containerID="736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08"
Jul 09 02:37:11.845254 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.845184    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08"} err="failed to get container status \"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08\": rpc error: code = NotFound desc = could not find container \"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08\": container with ID starting with 736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08 not found: ID does not exist"
Jul 09 02:37:11.845254 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.845210    2578 scope.go:117] "RemoveContainer" containerID="bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3"
Jul 09 02:37:11.845559 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.845539    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3"} err="failed to get container status \"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3\": rpc error: code = NotFound desc = could not find container \"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3\": container with ID starting with bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3 not found: ID does not exist"
Jul 09 02:37:11.845631 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.845560    2578 scope.go:117] "RemoveContainer" containerID="4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800"
Jul 09 02:37:11.845823 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.845805    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800"} err="failed to get container status \"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800\": rpc error: code = NotFound desc = could not find container \"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800\": container with ID starting with 4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800 not found: ID does not exist"
Jul 09 02:37:11.845896 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.845825    2578 scope.go:117] "RemoveContainer" containerID="e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82"
Jul 09 02:37:11.846038 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846013    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82"} err="failed to get container status \"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82\": rpc error: code = NotFound desc = could not find container \"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82\": container with ID starting with e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82 not found: ID does not exist"
Jul 09 02:37:11.846095 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846039    2578 scope.go:117] "RemoveContainer" containerID="5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92"
Jul 09 02:37:11.846261 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846241    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92"} err="failed to get container status \"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92\": rpc error: code = NotFound desc = could not find container \"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92\": container with ID starting with 5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92 not found: ID does not exist"
Jul 09 02:37:11.846355 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846262    2578 scope.go:117] "RemoveContainer" containerID="0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a"
Jul 09 02:37:11.846530 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846512    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a"} err="failed to get container status \"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a\": rpc error: code = NotFound desc = could not find container \"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a\": container with ID starting with 0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a not found: ID does not exist"
Jul 09 02:37:11.846615 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846531    2578 scope.go:117] "RemoveContainer" containerID="288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91"
Jul 09 02:37:11.846615 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846522    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 02:37:11.846780 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846763    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91"} err="failed to get container status \"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91\": rpc error: code = NotFound desc = could not find container \"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91\": container with ID starting with 288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91 not found: ID does not exist"
Jul 09 02:37:11.846824 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846781    2578 scope.go:117] "RemoveContainer" containerID="736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08"
Jul 09 02:37:11.846872 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846858    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="kube-rbac-proxy-thanos"
Jul 09 02:37:11.846914 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846877    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="kube-rbac-proxy-thanos"
Jul 09 02:37:11.846914 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846891    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="config-reloader"
Jul 09 02:37:11.846914 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846901    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="config-reloader"
Jul 09 02:37:11.846914 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846911    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="thanos-sidecar"
Jul 09 02:37:11.847043 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846920    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="thanos-sidecar"
Jul 09 02:37:11.847043 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846934    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="init-config-reloader"
Jul 09 02:37:11.847043 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846943    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="init-config-reloader"
Jul 09 02:37:11.847043 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846964    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="kube-rbac-proxy-web"
Jul 09 02:37:11.847043 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846971    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="kube-rbac-proxy-web"
Jul 09 02:37:11.847043 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846982    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="prometheus"
Jul 09 02:37:11.847043 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846987    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="prometheus"
Jul 09 02:37:11.847043 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846993    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="kube-rbac-proxy"
Jul 09 02:37:11.847043 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.846999    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="kube-rbac-proxy"
Jul 09 02:37:11.847043 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.847019    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08"} err="failed to get container status \"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08\": rpc error: code = NotFound desc = could not find container \"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08\": container with ID starting with 736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08 not found: ID does not exist"
Jul 09 02:37:11.847043 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.847040    2578 scope.go:117] "RemoveContainer" containerID="bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3"
Jul 09 02:37:11.847043 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.847048    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="kube-rbac-proxy-thanos"
Jul 09 02:37:11.847495 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.847057    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="kube-rbac-proxy-web"
Jul 09 02:37:11.847495 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.847064    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="thanos-sidecar"
Jul 09 02:37:11.847495 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.847070    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="kube-rbac-proxy"
Jul 09 02:37:11.847495 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.847077    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="config-reloader"
Jul 09 02:37:11.847495 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.847083    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" containerName="prometheus"
Jul 09 02:37:11.847495 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.847258    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3"} err="failed to get container status \"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3\": rpc error: code = NotFound desc = could not find container \"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3\": container with ID starting with bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3 not found: ID does not exist"
Jul 09 02:37:11.847495 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.847283    2578 scope.go:117] "RemoveContainer" containerID="4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800"
Jul 09 02:37:11.847708 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.847520    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800"} err="failed to get container status \"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800\": rpc error: code = NotFound desc = could not find container \"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800\": container with ID starting with 4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800 not found: ID does not exist"
Jul 09 02:37:11.847708 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.847535    2578 scope.go:117] "RemoveContainer" containerID="e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82"
Jul 09 02:37:11.847772 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.847723    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82"} err="failed to get container status \"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82\": rpc error: code = NotFound desc = could not find container \"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82\": container with ID starting with e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82 not found: ID does not exist"
Jul 09 02:37:11.847772 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.847736    2578 scope.go:117] "RemoveContainer" containerID="5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92"
Jul 09 02:37:11.847978 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.847954    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92"} err="failed to get container status \"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92\": rpc error: code = NotFound desc = could not find container \"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92\": container with ID starting with 5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92 not found: ID does not exist"
Jul 09 02:37:11.848018 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.847981    2578 scope.go:117] "RemoveContainer" containerID="0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a"
Jul 09 02:37:11.848209 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.848191    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a"} err="failed to get container status \"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a\": rpc error: code = NotFound desc = could not find container \"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a\": container with ID starting with 0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a not found: ID does not exist"
Jul 09 02:37:11.848255 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.848210    2578 scope.go:117] "RemoveContainer" containerID="288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91"
Jul 09 02:37:11.848433 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.848413    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91"} err="failed to get container status \"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91\": rpc error: code = NotFound desc = could not find container \"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91\": container with ID starting with 288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91 not found: ID does not exist"
Jul 09 02:37:11.848485 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.848433    2578 scope.go:117] "RemoveContainer" containerID="736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08"
Jul 09 02:37:11.848631 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.848617    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08"} err="failed to get container status \"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08\": rpc error: code = NotFound desc = could not find container \"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08\": container with ID starting with 736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08 not found: ID does not exist"
Jul 09 02:37:11.848676 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.848631    2578 scope.go:117] "RemoveContainer" containerID="bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3"
Jul 09 02:37:11.848833 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.848811    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3"} err="failed to get container status \"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3\": rpc error: code = NotFound desc = could not find container \"bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3\": container with ID starting with bb3cde9ee7867ed0a9cccf06b6a97f40a372d4521a8a3e04d72e504250623bd3 not found: ID does not exist"
Jul 09 02:37:11.848896 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.848834    2578 scope.go:117] "RemoveContainer" containerID="4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800"
Jul 09 02:37:11.849079 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.849061    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800"} err="failed to get container status \"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800\": rpc error: code = NotFound desc = could not find container \"4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800\": container with ID starting with 4244abd2ae064be78f70403f5c6491e00b66d8b07711cbfc4d6579b9546a2800 not found: ID does not exist"
Jul 09 02:37:11.849132 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.849080    2578 scope.go:117] "RemoveContainer" containerID="e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82"
Jul 09 02:37:11.849290 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.849272    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82"} err="failed to get container status \"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82\": rpc error: code = NotFound desc = could not find container \"e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82\": container with ID starting with e1e4ab0e175d97333aaf116a6a33cc26559a598431ed479374d59d612f364a82 not found: ID does not exist"
Jul 09 02:37:11.849425 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.849291    2578 scope.go:117] "RemoveContainer" containerID="5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92"
Jul 09 02:37:11.849513 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.849495    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92"} err="failed to get container status \"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92\": rpc error: code = NotFound desc = could not find container \"5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92\": container with ID starting with 5839ad455a76427538fec97e2563ef474ed276f04bebf16c71916da2a6cfbe92 not found: ID does not exist"
Jul 09 02:37:11.849551 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.849514    2578 scope.go:117] "RemoveContainer" containerID="0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a"
Jul 09 02:37:11.849753 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.849736    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a"} err="failed to get container status \"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a\": rpc error: code = NotFound desc = could not find container \"0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a\": container with ID starting with 0c7969729fd71a7c27fa2836744b8fb0c7ca447a970394486e37c9ed28325f8a not found: ID does not exist"
Jul 09 02:37:11.849753 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.849752    2578 scope.go:117] "RemoveContainer" containerID="288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91"
Jul 09 02:37:11.849956 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.849942    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91"} err="failed to get container status \"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91\": rpc error: code = NotFound desc = could not find container \"288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91\": container with ID starting with 288a71b833cfb43eb40aa4915ee2f24901ad83d3ba4a2cb2dce584b4c52a0a91 not found: ID does not exist"
Jul 09 02:37:11.849995 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.849956    2578 scope.go:117] "RemoveContainer" containerID="736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08"
Jul 09 02:37:11.850145 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.850128    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08"} err="failed to get container status \"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08\": rpc error: code = NotFound desc = could not find container \"736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08\": container with ID starting with 736b7f7ae3a2b3fa60b2bb29cc0a1dc00223902295f6ef734cba667229977b08 not found: ID does not exist"
Jul 09 02:37:11.852308 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.852286    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.854605 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.854561    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 09 02:37:11.854605 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.854598    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 09 02:37:11.854758 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.854612    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5jfhb\""
Jul 09 02:37:11.854758 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.854629    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 09 02:37:11.854861 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.854784    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 09 02:37:11.854861 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.854821    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 09 02:37:11.854861 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.854833    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-45rc09dnngj0q\""
Jul 09 02:37:11.855385 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.855368    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 09 02:37:11.855437 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.855395    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 09 02:37:11.855437 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.855371    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 09 02:37:11.855533 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.855450    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 09 02:37:11.855645 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.855633    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 09 02:37:11.855994 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.855974    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 09 02:37:11.856093 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.855996    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 09 02:37:11.858055 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858039    2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-web-config\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858055 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858056    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-db\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858162 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858067    2578 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858162 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858082    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858162 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858085    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 09 02:37:11.858162 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858091    2578 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-grpc-tls\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858162 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858123    2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858162 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858134    2578 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-metrics-client-certs\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858162 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858143    2578 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-config\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858162 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858151    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78bqw\" (UniqueName: \"kubernetes.io/projected/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-kube-api-access-78bqw\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858162 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858160    2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858456 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858168    2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-config-out\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858456 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858177    2578 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858456 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858191    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858456 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858202    2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-tls-assets\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858456 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858210    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858456 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858218    2578 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-kube-rbac-proxy\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858456 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858226    2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858456 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858235    2578 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-metrics-client-ca\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.858456 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.858243    2578 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:37:11.860274 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.860253    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 09 02:37:11.866045 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.866018    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 02:37:11.958860 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.958824    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-config\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959060 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.958870    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959060 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.958889    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-web-config\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959060 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.958909    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959060 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.958960    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959060 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.959003    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959060 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.959046    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959297 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.959075    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/881efdbb-3e3c-456b-b0cc-254642d3719f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959297 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.959096    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959297 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.959140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959297 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.959160    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959297 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.959178    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959297 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.959204    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959297 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.959223    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959297 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.959237    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/881efdbb-3e3c-456b-b0cc-254642d3719f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959297 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.959262    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/881efdbb-3e3c-456b-b0cc-254642d3719f-config-out\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959297 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.959277    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959297 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.959294    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5zpk\" (UniqueName: \"kubernetes.io/projected/881efdbb-3e3c-456b-b0cc-254642d3719f-kube-api-access-n5zpk\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959712 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.959311    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:11.959712 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:11.959345    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.060422 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060287    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/881efdbb-3e3c-456b-b0cc-254642d3719f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.060422 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060383    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/881efdbb-3e3c-456b-b0cc-254642d3719f-config-out\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.060422 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060405    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.060674 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060434    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5zpk\" (UniqueName: \"kubernetes.io/projected/881efdbb-3e3c-456b-b0cc-254642d3719f-kube-api-access-n5zpk\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.060674 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060575    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.060674 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060612    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.060674 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-config\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.060862 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060699    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.060862 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060731    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-web-config\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.060862 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060761    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.060862 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060794    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.060862 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060828    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.061113 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060864    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.061113 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060895    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/881efdbb-3e3c-456b-b0cc-254642d3719f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.061113 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060922    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.061113 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060966    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.061113 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.060992    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.061113 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.061019    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.061113 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.061057    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.061113 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.061063    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.061113 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.061090    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.061832 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.061807    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.062055 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.062032    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.062115 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.062099    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/881efdbb-3e3c-456b-b0cc-254642d3719f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.062934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.062601    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.063667 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.063352    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/881efdbb-3e3c-456b-b0cc-254642d3719f-config-out\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.063667 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.063423    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-config\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.063667 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.063466    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/881efdbb-3e3c-456b-b0cc-254642d3719f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.063667 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.063470    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.064652 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.064395    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.065191 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.064865    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.065191 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.065069    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.065191 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.065072    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.065191 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.065172    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/881efdbb-3e3c-456b-b0cc-254642d3719f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.065421 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.065348    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.065905 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.065878    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.066047 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.066031    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-web-config\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.066240 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.066224    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.067001 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.066980    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/881efdbb-3e3c-456b-b0cc-254642d3719f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.068013 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.067996    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5zpk\" (UniqueName: \"kubernetes.io/projected/881efdbb-3e3c-456b-b0cc-254642d3719f-kube-api-access-n5zpk\") pod \"prometheus-k8s-0\" (UID: \"881efdbb-3e3c-456b-b0cc-254642d3719f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.115966 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.115935    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c" path="/var/lib/kubelet/pods/1f35f9e9-3eb7-4bf4-b9e4-c67ffdc9045c/volumes"
Jul 09 02:37:12.162310 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.162279    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:37:12.295148 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.295070    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 02:37:12.297642 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:37:12.297620    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881efdbb_3e3c_456b_b0cc_254642d3719f.slice/crio-8ad7a9056b2f643dd281ec0dedeba22150bbd7a02a828ec7a13c0ff1253e97b1 WatchSource:0}: Error finding container 8ad7a9056b2f643dd281ec0dedeba22150bbd7a02a828ec7a13c0ff1253e97b1: Status 404 returned error can't find the container with id 8ad7a9056b2f643dd281ec0dedeba22150bbd7a02a828ec7a13c0ff1253e97b1
Jul 09 02:37:12.797497 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.797460    2578 generic.go:356] "Generic (PLEG): container finished" podID="881efdbb-3e3c-456b-b0cc-254642d3719f" containerID="fad749cf552e364c55f370a9058a8d71584a7ece78dc4c16e32645fb41cd86de" exitCode=0
Jul 09 02:37:12.797834 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.797544    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"881efdbb-3e3c-456b-b0cc-254642d3719f","Type":"ContainerDied","Data":"fad749cf552e364c55f370a9058a8d71584a7ece78dc4c16e32645fb41cd86de"}
Jul 09 02:37:12.797834 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:12.797607    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"881efdbb-3e3c-456b-b0cc-254642d3719f","Type":"ContainerStarted","Data":"8ad7a9056b2f643dd281ec0dedeba22150bbd7a02a828ec7a13c0ff1253e97b1"}
Jul 09 02:37:13.804605 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:13.804571    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"881efdbb-3e3c-456b-b0cc-254642d3719f","Type":"ContainerStarted","Data":"9944062bb169c04cc04286c36db805560aac7b49c101d9dfb6efeab4b427f022"}
Jul 09 02:37:13.804983 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:13.804635    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"881efdbb-3e3c-456b-b0cc-254642d3719f","Type":"ContainerStarted","Data":"95e4e3570c913878a45159be13660b7b6ead0967d530055c7de327283385e6f6"}
Jul 09 02:37:13.804983 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:13.804671    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"881efdbb-3e3c-456b-b0cc-254642d3719f","Type":"ContainerStarted","Data":"cc10c0250f227ab17dccd4ee61d64a4cfd11397839031d75a5a562f6b31d3f57"}
Jul 09 02:37:13.804983 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:13.804699    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"881efdbb-3e3c-456b-b0cc-254642d3719f","Type":"ContainerStarted","Data":"7d0e39a321fcb84ea151d48a9296050eeed62fa2615db252ea126e0039cdcded"}
Jul 09 02:37:13.804983 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:13.804726    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"881efdbb-3e3c-456b-b0cc-254642d3719f","Type":"ContainerStarted","Data":"176c55eed1b379fb5bfc076fcc6e09aa583fdc45a7ce9231410360bc4dddc5d8"}
Jul 09 02:37:13.804983 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:13.804759    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"881efdbb-3e3c-456b-b0cc-254642d3719f","Type":"ContainerStarted","Data":"99093a1c05f2d652109a5b24d59ec751e1aee4dc2c063707a59360347e3542d1"}
Jul 09 02:37:13.832712 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:13.832652    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.81525935 podStartE2EDuration="2.832631472s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.372122ms" isStatefulPod=false podCreationTimestamp="2026-07-09 02:37:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:37:13.830775957 +0000 UTC m=+148.224579382" watchObservedRunningTime="2026-07-09 02:37:13.832631472 +0000 UTC m=+148.226434886"
Jul 09 02:37:17.163184 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:37:17.163150    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:38:12.162926 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:12.162891    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:38:12.177949 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:12.177925    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:38:13.034611 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:13.034584    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 02:38:23.005109 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:23.005072    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-52stn"]
Jul 09 02:38:23.008526 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:23.008507    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-52stn"
Jul 09 02:38:23.010527 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:23.010508    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 09 02:38:23.010959 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:23.010943    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-wvv6m\""
Jul 09 02:38:23.011018 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:23.010965    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 09 02:38:23.016976 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:23.016954    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-52stn"]
Jul 09 02:38:23.124882 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:23.124854    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/29b279ae-7e74-4091-b28b-ffdeeb237f4b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-52stn\" (UID: \"29b279ae-7e74-4091-b28b-ffdeeb237f4b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-52stn"
Jul 09 02:38:23.125011 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:23.124900    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnqvf\" (UniqueName: \"kubernetes.io/projected/29b279ae-7e74-4091-b28b-ffdeeb237f4b-kube-api-access-nnqvf\") pod \"cert-manager-cainjector-559cdd8c87-52stn\" (UID: \"29b279ae-7e74-4091-b28b-ffdeeb237f4b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-52stn"
Jul 09 02:38:23.226270 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:23.226230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/29b279ae-7e74-4091-b28b-ffdeeb237f4b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-52stn\" (UID: \"29b279ae-7e74-4091-b28b-ffdeeb237f4b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-52stn"
Jul 09 02:38:23.226414 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:23.226304    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnqvf\" (UniqueName: \"kubernetes.io/projected/29b279ae-7e74-4091-b28b-ffdeeb237f4b-kube-api-access-nnqvf\") pod \"cert-manager-cainjector-559cdd8c87-52stn\" (UID: \"29b279ae-7e74-4091-b28b-ffdeeb237f4b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-52stn"
Jul 09 02:38:23.233603 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:23.233583    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/29b279ae-7e74-4091-b28b-ffdeeb237f4b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-52stn\" (UID: \"29b279ae-7e74-4091-b28b-ffdeeb237f4b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-52stn"
Jul 09 02:38:23.233603 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:23.233592    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnqvf\" (UniqueName: \"kubernetes.io/projected/29b279ae-7e74-4091-b28b-ffdeeb237f4b-kube-api-access-nnqvf\") pod \"cert-manager-cainjector-559cdd8c87-52stn\" (UID: \"29b279ae-7e74-4091-b28b-ffdeeb237f4b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-52stn"
Jul 09 02:38:23.329079 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:23.329011    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-52stn"
Jul 09 02:38:23.452016 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:23.451969    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-52stn"]
Jul 09 02:38:23.454806 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:38:23.454770    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29b279ae_7e74_4091_b28b_ffdeeb237f4b.slice/crio-c0bac5674b3aae7a1d625e842028c223f49a43f4e58215b54e31c2d6f711ce6f WatchSource:0}: Error finding container c0bac5674b3aae7a1d625e842028c223f49a43f4e58215b54e31c2d6f711ce6f: Status 404 returned error can't find the container with id c0bac5674b3aae7a1d625e842028c223f49a43f4e58215b54e31c2d6f711ce6f
Jul 09 02:38:24.059694 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:24.059660    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-52stn" event={"ID":"29b279ae-7e74-4091-b28b-ffdeeb237f4b","Type":"ContainerStarted","Data":"c0bac5674b3aae7a1d625e842028c223f49a43f4e58215b54e31c2d6f711ce6f"}
Jul 09 02:38:27.074446 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:27.074337    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-52stn" event={"ID":"29b279ae-7e74-4091-b28b-ffdeeb237f4b","Type":"ContainerStarted","Data":"cfee6ee8bcfaf1240d91002ef45ca71e1c85094907bb95e9d3d25bdffe3a3fd9"}
Jul 09 02:38:27.090075 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:27.089993    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-52stn" podStartSLOduration=1.724106922 podStartE2EDuration="5.089972856s" totalImagesPullingTime="3.365865934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:38:27.088804257 +0000 UTC m=+221.482607673" watchObservedRunningTime="2026-07-09 02:38:27.089972856 +0000 UTC m=+221.483776269"
Jul 09 02:38:58.883430 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:58.883386    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"]
Jul 09 02:38:58.890013 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:58.889993    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:58.892450 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:58.892426    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 09 02:38:58.892581 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:58.892555    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 09 02:38:58.892581 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:58.892480    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 09 02:38:58.892700 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:58.892558    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 09 02:38:58.892700 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:58.892429    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-d59qt\""
Jul 09 02:38:58.909051 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:58.909028    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"]
Jul 09 02:38:59.011052 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.011006    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1e0e49f3-2077-43ad-b073-a2d38a355d40-manifests\") pod \"opendatahub-operator-controller-manager-75df5ddb9c-m5mtm\" (UID: \"1e0e49f3-2077-43ad-b073-a2d38a355d40\") " pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:59.011052 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.011047    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e0e49f3-2077-43ad-b073-a2d38a355d40-webhook-cert\") pod \"opendatahub-operator-controller-manager-75df5ddb9c-m5mtm\" (UID: \"1e0e49f3-2077-43ad-b073-a2d38a355d40\") " pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:59.011256 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.011157    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt6fn\" (UniqueName: \"kubernetes.io/projected/1e0e49f3-2077-43ad-b073-a2d38a355d40-kube-api-access-vt6fn\") pod \"opendatahub-operator-controller-manager-75df5ddb9c-m5mtm\" (UID: \"1e0e49f3-2077-43ad-b073-a2d38a355d40\") " pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:59.011256 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.011201    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e0e49f3-2077-43ad-b073-a2d38a355d40-tmp\") pod \"opendatahub-operator-controller-manager-75df5ddb9c-m5mtm\" (UID: \"1e0e49f3-2077-43ad-b073-a2d38a355d40\") " pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:59.011256 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.011232    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1e0e49f3-2077-43ad-b073-a2d38a355d40-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75df5ddb9c-m5mtm\" (UID: \"1e0e49f3-2077-43ad-b073-a2d38a355d40\") " pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:59.111678 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.111638    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6fn\" (UniqueName: \"kubernetes.io/projected/1e0e49f3-2077-43ad-b073-a2d38a355d40-kube-api-access-vt6fn\") pod \"opendatahub-operator-controller-manager-75df5ddb9c-m5mtm\" (UID: \"1e0e49f3-2077-43ad-b073-a2d38a355d40\") " pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:59.111678 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.111683    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e0e49f3-2077-43ad-b073-a2d38a355d40-tmp\") pod \"opendatahub-operator-controller-manager-75df5ddb9c-m5mtm\" (UID: \"1e0e49f3-2077-43ad-b073-a2d38a355d40\") " pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:59.111927 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.111730    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1e0e49f3-2077-43ad-b073-a2d38a355d40-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75df5ddb9c-m5mtm\" (UID: \"1e0e49f3-2077-43ad-b073-a2d38a355d40\") " pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:59.111927 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.111771    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1e0e49f3-2077-43ad-b073-a2d38a355d40-manifests\") pod \"opendatahub-operator-controller-manager-75df5ddb9c-m5mtm\" (UID: \"1e0e49f3-2077-43ad-b073-a2d38a355d40\") " pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:59.111927 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.111792    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e0e49f3-2077-43ad-b073-a2d38a355d40-webhook-cert\") pod \"opendatahub-operator-controller-manager-75df5ddb9c-m5mtm\" (UID: \"1e0e49f3-2077-43ad-b073-a2d38a355d40\") " pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:59.112159 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.112132    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1e0e49f3-2077-43ad-b073-a2d38a355d40-manifests\") pod \"opendatahub-operator-controller-manager-75df5ddb9c-m5mtm\" (UID: \"1e0e49f3-2077-43ad-b073-a2d38a355d40\") " pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:59.113981 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.113959    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e0e49f3-2077-43ad-b073-a2d38a355d40-tmp\") pod \"opendatahub-operator-controller-manager-75df5ddb9c-m5mtm\" (UID: \"1e0e49f3-2077-43ad-b073-a2d38a355d40\") " pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:59.114185 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.114165    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e0e49f3-2077-43ad-b073-a2d38a355d40-webhook-cert\") pod \"opendatahub-operator-controller-manager-75df5ddb9c-m5mtm\" (UID: \"1e0e49f3-2077-43ad-b073-a2d38a355d40\") " pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:59.114233 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.114174    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1e0e49f3-2077-43ad-b073-a2d38a355d40-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75df5ddb9c-m5mtm\" (UID: \"1e0e49f3-2077-43ad-b073-a2d38a355d40\") " pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:59.127346 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.127301    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt6fn\" (UniqueName: \"kubernetes.io/projected/1e0e49f3-2077-43ad-b073-a2d38a355d40-kube-api-access-vt6fn\") pod \"opendatahub-operator-controller-manager-75df5ddb9c-m5mtm\" (UID: \"1e0e49f3-2077-43ad-b073-a2d38a355d40\") " pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:59.200907 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.200781    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:38:59.335647 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:38:59.335549    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"]
Jul 09 02:38:59.338603 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:38:59.338577    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e0e49f3_2077_43ad_b073_a2d38a355d40.slice/crio-06e7152876e6e377ea8225fe37e21d921a03572023ef0c1ddb42fef42c0d3c4a WatchSource:0}: Error finding container 06e7152876e6e377ea8225fe37e21d921a03572023ef0c1ddb42fef42c0d3c4a: Status 404 returned error can't find the container with id 06e7152876e6e377ea8225fe37e21d921a03572023ef0c1ddb42fef42c0d3c4a
Jul 09 02:39:00.193516 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:00.193469    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm" event={"ID":"1e0e49f3-2077-43ad-b073-a2d38a355d40","Type":"ContainerStarted","Data":"06e7152876e6e377ea8225fe37e21d921a03572023ef0c1ddb42fef42c0d3c4a"}
Jul 09 02:39:03.207571 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:03.207539    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm" event={"ID":"1e0e49f3-2077-43ad-b073-a2d38a355d40","Type":"ContainerStarted","Data":"ad73d4c484eb5f2c57344e79967b44710f1ab8dd674dbcc539b76da97c337e19"}
Jul 09 02:39:04.212922 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:04.212883    2578 generic.go:356] "Generic (PLEG): container finished" podID="1e0e49f3-2077-43ad-b073-a2d38a355d40" containerID="ad73d4c484eb5f2c57344e79967b44710f1ab8dd674dbcc539b76da97c337e19" exitCode=0
Jul 09 02:39:04.213304 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:04.212959    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm" event={"ID":"1e0e49f3-2077-43ad-b073-a2d38a355d40","Type":"ContainerDied","Data":"ad73d4c484eb5f2c57344e79967b44710f1ab8dd674dbcc539b76da97c337e19"}
Jul 09 02:39:05.219063 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:05.219029    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm" event={"ID":"1e0e49f3-2077-43ad-b073-a2d38a355d40","Type":"ContainerStarted","Data":"5a59db23d60371d596d25c65fa597622c20971fe6d5be10ccce26ec11d2871c4"}
Jul 09 02:39:05.219444 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:05.219147    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:39:05.243070 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:05.243015    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm" podStartSLOduration=2.738171357 podStartE2EDuration="7.242999985s" totalImagesPullingTime="3.4246681s" totalInitContainerRuntime="1.080160528s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:38:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:39:05.241304702 +0000 UTC m=+259.635108127" watchObservedRunningTime="2026-07-09 02:39:05.242999985 +0000 UTC m=+259.636803398"
Jul 09 02:39:16.224411 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:16.224336    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-75df5ddb9c-m5mtm"
Jul 09 02:39:23.848575 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:23.848543    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-75465c7b46-h57n5"]
Jul 09 02:39:23.859425 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:23.859405    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75465c7b46-h57n5"
Jul 09 02:39:23.862762 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:23.862736    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-2szn9\""
Jul 09 02:39:23.862885 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:23.862744    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 09 02:39:23.862885 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:23.862868    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 09 02:39:23.863018 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:23.862801    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 09 02:39:23.863018 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:23.862962    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 09 02:39:23.863108 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:23.863059    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75465c7b46-h57n5"]
Jul 09 02:39:24.029737 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.029699    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da8b0328-7d6f-497b-a0df-8eec6c05632d-tmp\") pod \"kube-auth-proxy-75465c7b46-h57n5\" (UID: \"da8b0328-7d6f-497b-a0df-8eec6c05632d\") " pod="openshift-ingress/kube-auth-proxy-75465c7b46-h57n5"
Jul 09 02:39:24.029895 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.029758    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmnds\" (UniqueName: \"kubernetes.io/projected/da8b0328-7d6f-497b-a0df-8eec6c05632d-kube-api-access-fmnds\") pod \"kube-auth-proxy-75465c7b46-h57n5\" (UID: \"da8b0328-7d6f-497b-a0df-8eec6c05632d\") " pod="openshift-ingress/kube-auth-proxy-75465c7b46-h57n5"
Jul 09 02:39:24.029895 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.029799    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da8b0328-7d6f-497b-a0df-8eec6c05632d-tls-certs\") pod \"kube-auth-proxy-75465c7b46-h57n5\" (UID: \"da8b0328-7d6f-497b-a0df-8eec6c05632d\") " pod="openshift-ingress/kube-auth-proxy-75465c7b46-h57n5"
Jul 09 02:39:24.130349 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.130303    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da8b0328-7d6f-497b-a0df-8eec6c05632d-tmp\") pod \"kube-auth-proxy-75465c7b46-h57n5\" (UID: \"da8b0328-7d6f-497b-a0df-8eec6c05632d\") " pod="openshift-ingress/kube-auth-proxy-75465c7b46-h57n5"
Jul 09 02:39:24.130526 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.130365    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmnds\" (UniqueName: \"kubernetes.io/projected/da8b0328-7d6f-497b-a0df-8eec6c05632d-kube-api-access-fmnds\") pod \"kube-auth-proxy-75465c7b46-h57n5\" (UID: \"da8b0328-7d6f-497b-a0df-8eec6c05632d\") " pod="openshift-ingress/kube-auth-proxy-75465c7b46-h57n5"
Jul 09 02:39:24.130526 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.130399    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da8b0328-7d6f-497b-a0df-8eec6c05632d-tls-certs\") pod \"kube-auth-proxy-75465c7b46-h57n5\" (UID: \"da8b0328-7d6f-497b-a0df-8eec6c05632d\") " pod="openshift-ingress/kube-auth-proxy-75465c7b46-h57n5"
Jul 09 02:39:24.132626 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.132604    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da8b0328-7d6f-497b-a0df-8eec6c05632d-tmp\") pod \"kube-auth-proxy-75465c7b46-h57n5\" (UID: \"da8b0328-7d6f-497b-a0df-8eec6c05632d\") " pod="openshift-ingress/kube-auth-proxy-75465c7b46-h57n5"
Jul 09 02:39:24.132958 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.132937    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da8b0328-7d6f-497b-a0df-8eec6c05632d-tls-certs\") pod \"kube-auth-proxy-75465c7b46-h57n5\" (UID: \"da8b0328-7d6f-497b-a0df-8eec6c05632d\") " pod="openshift-ingress/kube-auth-proxy-75465c7b46-h57n5"
Jul 09 02:39:24.137113 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.137091    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmnds\" (UniqueName: \"kubernetes.io/projected/da8b0328-7d6f-497b-a0df-8eec6c05632d-kube-api-access-fmnds\") pod \"kube-auth-proxy-75465c7b46-h57n5\" (UID: \"da8b0328-7d6f-497b-a0df-8eec6c05632d\") " pod="openshift-ingress/kube-auth-proxy-75465c7b46-h57n5"
Jul 09 02:39:24.170221 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.170200    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75465c7b46-h57n5"
Jul 09 02:39:24.328032 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.328006    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75465c7b46-h57n5"]
Jul 09 02:39:24.329201 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:39:24.329172    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda8b0328_7d6f_497b_a0df_8eec6c05632d.slice/crio-03e38030e2cfba6609e09db8a4e9670c09f7bd86f86170c1fdf562d40d9fab6c WatchSource:0}: Error finding container 03e38030e2cfba6609e09db8a4e9670c09f7bd86f86170c1fdf562d40d9fab6c: Status 404 returned error can't find the container with id 03e38030e2cfba6609e09db8a4e9670c09f7bd86f86170c1fdf562d40d9fab6c
Jul 09 02:39:24.917620 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.917588    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"]
Jul 09 02:39:24.921160 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.921129    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:24.923029 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.923003    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 09 02:39:24.923029 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.923024    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 09 02:39:24.923339 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.923306    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 09 02:39:24.923412 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.923346    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 09 02:39:24.923412 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.923375    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-tmbgc\""
Jul 09 02:39:24.932301 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:24.932280    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"]
Jul 09 02:39:25.036919 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.036886    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf49x\" (UniqueName: \"kubernetes.io/projected/3828cdfc-5f09-4e63-9a59-289813c27cdb-kube-api-access-zf49x\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.037073 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.036925    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-token\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.037073 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.036956    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3828cdfc-5f09-4e63-9a59-289813c27cdb-local-certs\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.037073 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.037013    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.037239 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.037090    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.037239 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.037170    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-cacerts\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.037239 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.037213    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.138095 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.138064    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.138248 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.138109    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-cacerts\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.138248 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.138133    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.138248 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.138163    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf49x\" (UniqueName: \"kubernetes.io/projected/3828cdfc-5f09-4e63-9a59-289813c27cdb-kube-api-access-zf49x\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.138248 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.138181    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-token\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.138248 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.138205    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3828cdfc-5f09-4e63-9a59-289813c27cdb-local-certs\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.138528 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.138418    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.139197 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.139142    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.140856 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.140822    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3828cdfc-5f09-4e63-9a59-289813c27cdb-local-certs\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.141149 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.141109    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-cacerts\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.141241 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.141170    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.141241 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.141170    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.146278 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.146250    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-token\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.146413 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.146388    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf49x\" (UniqueName: \"kubernetes.io/projected/3828cdfc-5f09-4e63-9a59-289813c27cdb-kube-api-access-zf49x\") pod \"istiod-openshift-gateway-7574459656-nfb95\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.233108 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.233049    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:25.294997 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.294930    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75465c7b46-h57n5" event={"ID":"da8b0328-7d6f-497b-a0df-8eec6c05632d","Type":"ContainerStarted","Data":"03e38030e2cfba6609e09db8a4e9670c09f7bd86f86170c1fdf562d40d9fab6c"}
Jul 09 02:39:25.397936 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:25.397854    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"]
Jul 09 02:39:25.400298 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:39:25.400269    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3828cdfc_5f09_4e63_9a59_289813c27cdb.slice/crio-89dc37850eb4250a6d4d8e834c4861ecea4fc7a43e9d2698bb2b0a93ce8ade2e WatchSource:0}: Error finding container 89dc37850eb4250a6d4d8e834c4861ecea4fc7a43e9d2698bb2b0a93ce8ade2e: Status 404 returned error can't find the container with id 89dc37850eb4250a6d4d8e834c4861ecea4fc7a43e9d2698bb2b0a93ce8ade2e
Jul 09 02:39:26.301867 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:26.301826    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95" event={"ID":"3828cdfc-5f09-4e63-9a59-289813c27cdb","Type":"ContainerStarted","Data":"89dc37850eb4250a6d4d8e834c4861ecea4fc7a43e9d2698bb2b0a93ce8ade2e"}
Jul 09 02:39:27.355231 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.355196    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"]
Jul 09 02:39:27.361588 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.361560    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.370062 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.370035    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"]
Jul 09 02:39:27.461643 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.461615    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/041897a1-32d2-4316-8f58-2b6fe74e92f9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.461821 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.461679    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/041897a1-32d2-4316-8f58-2b6fe74e92f9-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.461821 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.461713    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw799\" (UniqueName: \"kubernetes.io/projected/041897a1-32d2-4316-8f58-2b6fe74e92f9-kube-api-access-fw799\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.461821 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.461742    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/041897a1-32d2-4316-8f58-2b6fe74e92f9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.461821 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.461766    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/041897a1-32d2-4316-8f58-2b6fe74e92f9-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.461821 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.461813    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/041897a1-32d2-4316-8f58-2b6fe74e92f9-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.462022 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.461837    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/041897a1-32d2-4316-8f58-2b6fe74e92f9-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.562926 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.562889    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/041897a1-32d2-4316-8f58-2b6fe74e92f9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.563112 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.562946    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/041897a1-32d2-4316-8f58-2b6fe74e92f9-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.563112 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.562968    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw799\" (UniqueName: \"kubernetes.io/projected/041897a1-32d2-4316-8f58-2b6fe74e92f9-kube-api-access-fw799\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.563112 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.563022    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/041897a1-32d2-4316-8f58-2b6fe74e92f9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.563262 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.563157    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/041897a1-32d2-4316-8f58-2b6fe74e92f9-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.563262 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.563228    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/041897a1-32d2-4316-8f58-2b6fe74e92f9-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.563262 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.563259    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/041897a1-32d2-4316-8f58-2b6fe74e92f9-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.564015 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.563953    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/041897a1-32d2-4316-8f58-2b6fe74e92f9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.565865 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.565827    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/041897a1-32d2-4316-8f58-2b6fe74e92f9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.565865 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.565836    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/041897a1-32d2-4316-8f58-2b6fe74e92f9-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.566029 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.565860    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/041897a1-32d2-4316-8f58-2b6fe74e92f9-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.566398 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.566378    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/041897a1-32d2-4316-8f58-2b6fe74e92f9-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.574333 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.574285    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/041897a1-32d2-4316-8f58-2b6fe74e92f9-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.574449 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.574290    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw799\" (UniqueName: \"kubernetes.io/projected/041897a1-32d2-4316-8f58-2b6fe74e92f9-kube-api-access-fw799\") pod \"istiod-openshift-gateway-94bb8fbfd-2zf89\" (UID: \"041897a1-32d2-4316-8f58-2b6fe74e92f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:27.672023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:27.671934    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:28.314819 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:28.314784    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 02:39:28.314916 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:28.314849    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 02:39:28.390618 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:28.390557    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"]
Jul 09 02:39:28.394367 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:39:28.394336    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod041897a1_32d2_4316_8f58_2b6fe74e92f9.slice/crio-9071cdd10b028f5581b998c9413c286332f5bab6910ad603ff09d9cb803ec6dd WatchSource:0}: Error finding container 9071cdd10b028f5581b998c9413c286332f5bab6910ad603ff09d9cb803ec6dd: Status 404 returned error can't find the container with id 9071cdd10b028f5581b998c9413c286332f5bab6910ad603ff09d9cb803ec6dd
Jul 09 02:39:28.397197 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:28.397160    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 02:39:28.397265 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:28.397227    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 02:39:29.317933 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:29.317899    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95" event={"ID":"3828cdfc-5f09-4e63-9a59-289813c27cdb","Type":"ContainerStarted","Data":"733fcb69eaa85cba78d7156dce6ef7bf6266d8a980d621f1ad8e59e2087569b9"}
Jul 09 02:39:29.318167 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:29.318013    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:29.319680 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:29.319652    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89" event={"ID":"041897a1-32d2-4316-8f58-2b6fe74e92f9","Type":"ContainerStarted","Data":"9fd24dd387a984bd6550268e0c9b3a853591d7a9813d1fa1ce69d274af7e40aa"}
Jul 09 02:39:29.319813 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:29.319704    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:29.319813 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:29.319723    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89" event={"ID":"041897a1-32d2-4316-8f58-2b6fe74e92f9","Type":"ContainerStarted","Data":"9071cdd10b028f5581b998c9413c286332f5bab6910ad603ff09d9cb803ec6dd"}
Jul 09 02:39:29.320084 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:29.320061    2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-nfb95 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 09 02:39:29.320176 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:29.320105    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95" podUID="3828cdfc-5f09-4e63-9a59-289813c27cdb" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 02:39:29.321007 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:29.320984    2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-2zf89 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 09 02:39:29.321112 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:29.321022    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89" podUID="041897a1-32d2-4316-8f58-2b6fe74e92f9" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 02:39:29.321244 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:29.321222    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75465c7b46-h57n5" event={"ID":"da8b0328-7d6f-497b-a0df-8eec6c05632d","Type":"ContainerStarted","Data":"9043eb9bb868c10a8149d96013e6d942a458581948d4379e0436e0a593c63f06"}
Jul 09 02:39:29.406509 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:29.406457    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95" podStartSLOduration=2.494589022 podStartE2EDuration="5.406441153s" totalImagesPullingTime="2.911852131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:39:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:39:29.401041214 +0000 UTC m=+283.794844627" watchObservedRunningTime="2026-07-09 02:39:29.406441153 +0000 UTC m=+283.800244560"
Jul 09 02:39:29.451890 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:29.451840    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89" podStartSLOduration=2.451822704 podStartE2EDuration="2.451822704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:39:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:39:29.44694235 +0000 UTC m=+283.840745764" watchObservedRunningTime="2026-07-09 02:39:29.451822704 +0000 UTC m=+283.845626117"
Jul 09 02:39:29.495733 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:29.495681    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-75465c7b46-h57n5" podStartSLOduration=2.568994864 podStartE2EDuration="6.495664627s" totalImagesPullingTime="3.926669763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:39:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:39:29.494428735 +0000 UTC m=+283.888232148" watchObservedRunningTime="2026-07-09 02:39:29.495664627 +0000 UTC m=+283.889468040"
Jul 09 02:39:30.328614 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:30.328577    2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-2zf89 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 09 02:39:30.328897 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:30.328647    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89" podUID="041897a1-32d2-4316-8f58-2b6fe74e92f9" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 02:39:30.328897 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:30.328718    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:33.328504 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.328475    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2zf89"
Jul 09 02:39:33.381813 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.381784    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"]
Jul 09 02:39:33.381965 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.381942    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95" podUID="3828cdfc-5f09-4e63-9a59-289813c27cdb" containerName="discovery" containerID="cri-o://733fcb69eaa85cba78d7156dce6ef7bf6266d8a980d621f1ad8e59e2087569b9" gracePeriod=30
Jul 09 02:39:33.645865 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.645843    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:33.712074 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.712046    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-cacerts\") pod \"3828cdfc-5f09-4e63-9a59-289813c27cdb\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") "
Jul 09 02:39:33.712207 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.712102    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-csr-ca-configmap\") pod \"3828cdfc-5f09-4e63-9a59-289813c27cdb\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") "
Jul 09 02:39:33.712207 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.712139    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zf49x\" (UniqueName: \"kubernetes.io/projected/3828cdfc-5f09-4e63-9a59-289813c27cdb-kube-api-access-zf49x\") pod \"3828cdfc-5f09-4e63-9a59-289813c27cdb\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") "
Jul 09 02:39:33.712207 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.712162    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-token\") pod \"3828cdfc-5f09-4e63-9a59-289813c27cdb\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") "
Jul 09 02:39:33.712403 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.712238    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3828cdfc-5f09-4e63-9a59-289813c27cdb-local-certs\") pod \"3828cdfc-5f09-4e63-9a59-289813c27cdb\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") "
Jul 09 02:39:33.712403 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.712270    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-csr-dns-cert\") pod \"3828cdfc-5f09-4e63-9a59-289813c27cdb\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") "
Jul 09 02:39:33.712403 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.712356    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-kubeconfig\") pod \"3828cdfc-5f09-4e63-9a59-289813c27cdb\" (UID: \"3828cdfc-5f09-4e63-9a59-289813c27cdb\") "
Jul 09 02:39:33.712621 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.712574    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "3828cdfc-5f09-4e63-9a59-289813c27cdb" (UID: "3828cdfc-5f09-4e63-9a59-289813c27cdb"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 02:39:33.712972 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.712952    2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-csr-ca-configmap\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:39:33.714563 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.714538    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "3828cdfc-5f09-4e63-9a59-289813c27cdb" (UID: "3828cdfc-5f09-4e63-9a59-289813c27cdb"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 02:39:33.714657 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.714602    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "3828cdfc-5f09-4e63-9a59-289813c27cdb" (UID: "3828cdfc-5f09-4e63-9a59-289813c27cdb"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 02:39:33.714822 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.714788    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3828cdfc-5f09-4e63-9a59-289813c27cdb-local-certs" (OuterVolumeSpecName: "local-certs") pod "3828cdfc-5f09-4e63-9a59-289813c27cdb" (UID: "3828cdfc-5f09-4e63-9a59-289813c27cdb"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 02:39:33.714877 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.714864    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-cacerts" (OuterVolumeSpecName: "cacerts") pod "3828cdfc-5f09-4e63-9a59-289813c27cdb" (UID: "3828cdfc-5f09-4e63-9a59-289813c27cdb"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 02:39:33.714913 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.714864    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3828cdfc-5f09-4e63-9a59-289813c27cdb-kube-api-access-zf49x" (OuterVolumeSpecName: "kube-api-access-zf49x") pod "3828cdfc-5f09-4e63-9a59-289813c27cdb" (UID: "3828cdfc-5f09-4e63-9a59-289813c27cdb"). InnerVolumeSpecName "kube-api-access-zf49x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 02:39:33.714913 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.714873    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-token" (OuterVolumeSpecName: "istio-token") pod "3828cdfc-5f09-4e63-9a59-289813c27cdb" (UID: "3828cdfc-5f09-4e63-9a59-289813c27cdb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 02:39:33.814507 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.814478    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zf49x\" (UniqueName: \"kubernetes.io/projected/3828cdfc-5f09-4e63-9a59-289813c27cdb-kube-api-access-zf49x\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:39:33.814507 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.814502    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-token\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:39:33.814507 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.814512    2578 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3828cdfc-5f09-4e63-9a59-289813c27cdb-local-certs\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:39:33.814731 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.814524    2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-csr-dns-cert\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:39:33.814731 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.814532    2578 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-istio-kubeconfig\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:39:33.814731 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:33.814540    2578 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3828cdfc-5f09-4e63-9a59-289813c27cdb-cacerts\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:39:34.345943 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:34.345907    2578 generic.go:356] "Generic (PLEG): container finished" podID="3828cdfc-5f09-4e63-9a59-289813c27cdb" containerID="733fcb69eaa85cba78d7156dce6ef7bf6266d8a980d621f1ad8e59e2087569b9" exitCode=0
Jul 09 02:39:34.346402 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:34.345956    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"
Jul 09 02:39:34.346402 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:34.345989    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95" event={"ID":"3828cdfc-5f09-4e63-9a59-289813c27cdb","Type":"ContainerDied","Data":"733fcb69eaa85cba78d7156dce6ef7bf6266d8a980d621f1ad8e59e2087569b9"}
Jul 09 02:39:34.346402 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:34.346067    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfb95" event={"ID":"3828cdfc-5f09-4e63-9a59-289813c27cdb","Type":"ContainerDied","Data":"89dc37850eb4250a6d4d8e834c4861ecea4fc7a43e9d2698bb2b0a93ce8ade2e"}
Jul 09 02:39:34.346402 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:34.346101    2578 scope.go:117] "RemoveContainer" containerID="733fcb69eaa85cba78d7156dce6ef7bf6266d8a980d621f1ad8e59e2087569b9"
Jul 09 02:39:34.354030 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:34.354013    2578 scope.go:117] "RemoveContainer" containerID="733fcb69eaa85cba78d7156dce6ef7bf6266d8a980d621f1ad8e59e2087569b9"
Jul 09 02:39:34.354260 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:39:34.354242    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"733fcb69eaa85cba78d7156dce6ef7bf6266d8a980d621f1ad8e59e2087569b9\": container with ID starting with 733fcb69eaa85cba78d7156dce6ef7bf6266d8a980d621f1ad8e59e2087569b9 not found: ID does not exist" containerID="733fcb69eaa85cba78d7156dce6ef7bf6266d8a980d621f1ad8e59e2087569b9"
Jul 09 02:39:34.354306 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:34.354266    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"733fcb69eaa85cba78d7156dce6ef7bf6266d8a980d621f1ad8e59e2087569b9"} err="failed to get container status \"733fcb69eaa85cba78d7156dce6ef7bf6266d8a980d621f1ad8e59e2087569b9\": rpc error: code = NotFound desc = could not find container \"733fcb69eaa85cba78d7156dce6ef7bf6266d8a980d621f1ad8e59e2087569b9\": container with ID starting with 733fcb69eaa85cba78d7156dce6ef7bf6266d8a980d621f1ad8e59e2087569b9 not found: ID does not exist"
Jul 09 02:39:34.378719 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:34.378666    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"]
Jul 09 02:39:34.383928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:34.383909    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nfb95"]
Jul 09 02:39:36.116698 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:36.116663    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3828cdfc-5f09-4e63-9a59-289813c27cdb" path="/var/lib/kubelet/pods/3828cdfc-5f09-4e63-9a59-289813c27cdb/volumes"
Jul 09 02:39:45.949822 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:39:45.949797    2578 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 02:40:19.322668 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.322635    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ksx82"]
Jul 09 02:40:19.323110 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.322945    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3828cdfc-5f09-4e63-9a59-289813c27cdb" containerName="discovery"
Jul 09 02:40:19.323110 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.322956    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3828cdfc-5f09-4e63-9a59-289813c27cdb" containerName="discovery"
Jul 09 02:40:19.323110 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.323004    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3828cdfc-5f09-4e63-9a59-289813c27cdb" containerName="discovery"
Jul 09 02:40:19.325770 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.325746    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ksx82"
Jul 09 02:40:19.327951 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.327929    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 02:40:19.328083 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.327960    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 02:40:19.328422 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.328406    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-rfdhp\""
Jul 09 02:40:19.335939 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.335916    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ksx82"]
Jul 09 02:40:19.396007 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.395979    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfzqj\" (UniqueName: \"kubernetes.io/projected/6dfa5d13-6cc4-460d-99fe-4af854cc4ba4-kube-api-access-tfzqj\") pod \"kuadrant-operator-catalog-ksx82\" (UID: \"6dfa5d13-6cc4-460d-99fe-4af854cc4ba4\") " pod="kuadrant-system/kuadrant-operator-catalog-ksx82"
Jul 09 02:40:19.497460 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.497434    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfzqj\" (UniqueName: \"kubernetes.io/projected/6dfa5d13-6cc4-460d-99fe-4af854cc4ba4-kube-api-access-tfzqj\") pod \"kuadrant-operator-catalog-ksx82\" (UID: \"6dfa5d13-6cc4-460d-99fe-4af854cc4ba4\") " pod="kuadrant-system/kuadrant-operator-catalog-ksx82"
Jul 09 02:40:19.505043 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.505008    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfzqj\" (UniqueName: \"kubernetes.io/projected/6dfa5d13-6cc4-460d-99fe-4af854cc4ba4-kube-api-access-tfzqj\") pod \"kuadrant-operator-catalog-ksx82\" (UID: \"6dfa5d13-6cc4-460d-99fe-4af854cc4ba4\") " pod="kuadrant-system/kuadrant-operator-catalog-ksx82"
Jul 09 02:40:19.637462 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.637423    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ksx82"
Jul 09 02:40:19.687931 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.687895    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ksx82"]
Jul 09 02:40:19.761880 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.761713    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ksx82"]
Jul 09 02:40:19.764623 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:40:19.764593    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dfa5d13_6cc4_460d_99fe_4af854cc4ba4.slice/crio-46f6e491b445ec152a79b329ae0e2b02ac3bc8328cf019836404fba94bce7669 WatchSource:0}: Error finding container 46f6e491b445ec152a79b329ae0e2b02ac3bc8328cf019836404fba94bce7669: Status 404 returned error can't find the container with id 46f6e491b445ec152a79b329ae0e2b02ac3bc8328cf019836404fba94bce7669
Jul 09 02:40:19.765942 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.765923    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 02:40:19.898046 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.897975    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-htmvv"]
Jul 09 02:40:19.902455 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.902436    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-htmvv"
Jul 09 02:40:19.908110 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:19.908085    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-htmvv"]
Jul 09 02:40:20.001857 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:20.001832    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv7tx\" (UniqueName: \"kubernetes.io/projected/8d341bd6-f964-4887-a678-b50dd7788546-kube-api-access-bv7tx\") pod \"kuadrant-operator-catalog-htmvv\" (UID: \"8d341bd6-f964-4887-a678-b50dd7788546\") " pod="kuadrant-system/kuadrant-operator-catalog-htmvv"
Jul 09 02:40:20.102334 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:20.102294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv7tx\" (UniqueName: \"kubernetes.io/projected/8d341bd6-f964-4887-a678-b50dd7788546-kube-api-access-bv7tx\") pod \"kuadrant-operator-catalog-htmvv\" (UID: \"8d341bd6-f964-4887-a678-b50dd7788546\") " pod="kuadrant-system/kuadrant-operator-catalog-htmvv"
Jul 09 02:40:20.109403 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:20.109378    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv7tx\" (UniqueName: \"kubernetes.io/projected/8d341bd6-f964-4887-a678-b50dd7788546-kube-api-access-bv7tx\") pod \"kuadrant-operator-catalog-htmvv\" (UID: \"8d341bd6-f964-4887-a678-b50dd7788546\") " pod="kuadrant-system/kuadrant-operator-catalog-htmvv"
Jul 09 02:40:20.212450 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:20.212389    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-htmvv"
Jul 09 02:40:20.346452 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:20.346428    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-htmvv"]
Jul 09 02:40:20.348342 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:40:20.348302    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d341bd6_f964_4887_a678_b50dd7788546.slice/crio-3ba7f3d668fdd30bc1cdaaa486aa74b7e80aeb455a09170ca79446cc34611230 WatchSource:0}: Error finding container 3ba7f3d668fdd30bc1cdaaa486aa74b7e80aeb455a09170ca79446cc34611230: Status 404 returned error can't find the container with id 3ba7f3d668fdd30bc1cdaaa486aa74b7e80aeb455a09170ca79446cc34611230
Jul 09 02:40:20.528541 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:20.528459    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-htmvv" event={"ID":"8d341bd6-f964-4887-a678-b50dd7788546","Type":"ContainerStarted","Data":"3ba7f3d668fdd30bc1cdaaa486aa74b7e80aeb455a09170ca79446cc34611230"}
Jul 09 02:40:20.529783 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:20.529749    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ksx82" event={"ID":"6dfa5d13-6cc4-460d-99fe-4af854cc4ba4","Type":"ContainerStarted","Data":"46f6e491b445ec152a79b329ae0e2b02ac3bc8328cf019836404fba94bce7669"}
Jul 09 02:40:22.541134 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:22.541096    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ksx82" event={"ID":"6dfa5d13-6cc4-460d-99fe-4af854cc4ba4","Type":"ContainerStarted","Data":"34d15456221174db284d63b6cf58701e9286d33588e13ba526dc7625085bc4b1"}
Jul 09 02:40:22.541600 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:22.541134    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-ksx82" podUID="6dfa5d13-6cc4-460d-99fe-4af854cc4ba4" containerName="registry-server" containerID="cri-o://34d15456221174db284d63b6cf58701e9286d33588e13ba526dc7625085bc4b1" gracePeriod=2
Jul 09 02:40:22.542659 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:22.542628    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-htmvv" event={"ID":"8d341bd6-f964-4887-a678-b50dd7788546","Type":"ContainerStarted","Data":"a30b97649236e3189bb6087ced11809697799031a198e376a320ef587c61428d"}
Jul 09 02:40:22.556049 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:22.556000    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ksx82" podStartSLOduration=1.410135162 podStartE2EDuration="3.555986854s" totalImagesPullingTime="2.145851692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:40:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:40:22.554895363 +0000 UTC m=+336.948698776" watchObservedRunningTime="2026-07-09 02:40:22.555986854 +0000 UTC m=+336.949790267"
Jul 09 02:40:22.569276 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:22.569235    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-htmvv" podStartSLOduration=2.004098406 podStartE2EDuration="3.569222438s" totalImagesPullingTime="1.565124032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:40:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:40:22.568895344 +0000 UTC m=+336.962698768" watchObservedRunningTime="2026-07-09 02:40:22.569222438 +0000 UTC m=+336.963025851"
Jul 09 02:40:22.776181 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:22.776157    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ksx82"
Jul 09 02:40:22.828169 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:22.828110    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfzqj\" (UniqueName: \"kubernetes.io/projected/6dfa5d13-6cc4-460d-99fe-4af854cc4ba4-kube-api-access-tfzqj\") pod \"6dfa5d13-6cc4-460d-99fe-4af854cc4ba4\" (UID: \"6dfa5d13-6cc4-460d-99fe-4af854cc4ba4\") "
Jul 09 02:40:22.830124 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:22.830095    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dfa5d13-6cc4-460d-99fe-4af854cc4ba4-kube-api-access-tfzqj" (OuterVolumeSpecName: "kube-api-access-tfzqj") pod "6dfa5d13-6cc4-460d-99fe-4af854cc4ba4" (UID: "6dfa5d13-6cc4-460d-99fe-4af854cc4ba4"). InnerVolumeSpecName "kube-api-access-tfzqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 02:40:22.929843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:22.929809    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfzqj\" (UniqueName: \"kubernetes.io/projected/6dfa5d13-6cc4-460d-99fe-4af854cc4ba4-kube-api-access-tfzqj\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:40:23.548047 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:23.548013    2578 generic.go:356] "Generic (PLEG): container finished" podID="6dfa5d13-6cc4-460d-99fe-4af854cc4ba4" containerID="34d15456221174db284d63b6cf58701e9286d33588e13ba526dc7625085bc4b1" exitCode=0
Jul 09 02:40:23.548475 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:23.548055    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ksx82"
Jul 09 02:40:23.548475 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:23.548101    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ksx82" event={"ID":"6dfa5d13-6cc4-460d-99fe-4af854cc4ba4","Type":"ContainerDied","Data":"34d15456221174db284d63b6cf58701e9286d33588e13ba526dc7625085bc4b1"}
Jul 09 02:40:23.548475 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:23.548149    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ksx82" event={"ID":"6dfa5d13-6cc4-460d-99fe-4af854cc4ba4","Type":"ContainerDied","Data":"46f6e491b445ec152a79b329ae0e2b02ac3bc8328cf019836404fba94bce7669"}
Jul 09 02:40:23.548475 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:23.548173    2578 scope.go:117] "RemoveContainer" containerID="34d15456221174db284d63b6cf58701e9286d33588e13ba526dc7625085bc4b1"
Jul 09 02:40:23.556731 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:23.556703    2578 scope.go:117] "RemoveContainer" containerID="34d15456221174db284d63b6cf58701e9286d33588e13ba526dc7625085bc4b1"
Jul 09 02:40:23.556971 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:40:23.556953    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34d15456221174db284d63b6cf58701e9286d33588e13ba526dc7625085bc4b1\": container with ID starting with 34d15456221174db284d63b6cf58701e9286d33588e13ba526dc7625085bc4b1 not found: ID does not exist" containerID="34d15456221174db284d63b6cf58701e9286d33588e13ba526dc7625085bc4b1"
Jul 09 02:40:23.557036 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:23.556979    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34d15456221174db284d63b6cf58701e9286d33588e13ba526dc7625085bc4b1"} err="failed to get container status \"34d15456221174db284d63b6cf58701e9286d33588e13ba526dc7625085bc4b1\": rpc error: code = NotFound desc = could not find container \"34d15456221174db284d63b6cf58701e9286d33588e13ba526dc7625085bc4b1\": container with ID starting with 34d15456221174db284d63b6cf58701e9286d33588e13ba526dc7625085bc4b1 not found: ID does not exist"
Jul 09 02:40:23.567465 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:23.567443    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ksx82"]
Jul 09 02:40:23.571412 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:23.571388    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ksx82"]
Jul 09 02:40:24.117649 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:24.117614    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dfa5d13-6cc4-460d-99fe-4af854cc4ba4" path="/var/lib/kubelet/pods/6dfa5d13-6cc4-460d-99fe-4af854cc4ba4/volumes"
Jul 09 02:40:30.213380 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:30.213347    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-htmvv"
Jul 09 02:40:30.213774 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:30.213414    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-htmvv"
Jul 09 02:40:30.235699 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:30.235673    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-htmvv"
Jul 09 02:40:30.597887 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:30.597813    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-htmvv"
Jul 09 02:40:52.246989 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:52.246959    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jkb7l"]
Jul 09 02:40:52.247467 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:52.247334    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dfa5d13-6cc4-460d-99fe-4af854cc4ba4" containerName="registry-server"
Jul 09 02:40:52.247467 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:52.247346    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dfa5d13-6cc4-460d-99fe-4af854cc4ba4" containerName="registry-server"
Jul 09 02:40:52.247467 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:52.247416    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dfa5d13-6cc4-460d-99fe-4af854cc4ba4" containerName="registry-server"
Jul 09 02:40:52.251829 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:52.251811    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jkb7l"
Jul 09 02:40:52.255863 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:52.255838    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-vxkmp\""
Jul 09 02:40:52.262884 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:52.262864    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jkb7l"]
Jul 09 02:40:52.377679 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:52.377619    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzrjj\" (UniqueName: \"kubernetes.io/projected/82d746e1-23a4-4986-8afd-1d6265b4ebc1-kube-api-access-mzrjj\") pod \"authorino-operator-75cc8d5cb7-jkb7l\" (UID: \"82d746e1-23a4-4986-8afd-1d6265b4ebc1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jkb7l"
Jul 09 02:40:52.478785 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:52.478742    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzrjj\" (UniqueName: \"kubernetes.io/projected/82d746e1-23a4-4986-8afd-1d6265b4ebc1-kube-api-access-mzrjj\") pod \"authorino-operator-75cc8d5cb7-jkb7l\" (UID: \"82d746e1-23a4-4986-8afd-1d6265b4ebc1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jkb7l"
Jul 09 02:40:52.492881 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:52.492856    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzrjj\" (UniqueName: \"kubernetes.io/projected/82d746e1-23a4-4986-8afd-1d6265b4ebc1-kube-api-access-mzrjj\") pod \"authorino-operator-75cc8d5cb7-jkb7l\" (UID: \"82d746e1-23a4-4986-8afd-1d6265b4ebc1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jkb7l"
Jul 09 02:40:52.562255 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:52.562181    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jkb7l"
Jul 09 02:40:52.695540 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:52.695518    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jkb7l"]
Jul 09 02:40:52.697687 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:40:52.697660    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82d746e1_23a4_4986_8afd_1d6265b4ebc1.slice/crio-afb3f90cca6b8f1a90f34f36b474ea580b71d22e02a16773e39c53da4a2ff892 WatchSource:0}: Error finding container afb3f90cca6b8f1a90f34f36b474ea580b71d22e02a16773e39c53da4a2ff892: Status 404 returned error can't find the container with id afb3f90cca6b8f1a90f34f36b474ea580b71d22e02a16773e39c53da4a2ff892
Jul 09 02:40:53.674666 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:53.674624    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jkb7l" event={"ID":"82d746e1-23a4-4986-8afd-1d6265b4ebc1","Type":"ContainerStarted","Data":"afb3f90cca6b8f1a90f34f36b474ea580b71d22e02a16773e39c53da4a2ff892"}
Jul 09 02:40:55.490782 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:55.490747    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-77k6w"]
Jul 09 02:40:55.497733 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:55.497712    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77k6w"
Jul 09 02:40:55.500076 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:55.500051    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 09 02:40:55.500186 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:55.500088    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-ns4tp\""
Jul 09 02:40:55.505188 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:55.505031    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-77k6w"]
Jul 09 02:40:55.605235 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:55.605196    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bszcw\" (UniqueName: \"kubernetes.io/projected/df551de3-1753-4995-91f3-8198f0071445-kube-api-access-bszcw\") pod \"dns-operator-controller-manager-864dd78c4c-77k6w\" (UID: \"df551de3-1753-4995-91f3-8198f0071445\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77k6w"
Jul 09 02:40:55.684705 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:55.684666    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jkb7l" event={"ID":"82d746e1-23a4-4986-8afd-1d6265b4ebc1","Type":"ContainerStarted","Data":"c943940a453f8fbb3bc88bbe40260bf9f5f668ba6843ee8cbd3f537543526824"}
Jul 09 02:40:55.684890 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:55.684747    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jkb7l"
Jul 09 02:40:55.702875 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:55.702828    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jkb7l" podStartSLOduration=1.398726968 podStartE2EDuration="3.702812291s" totalImagesPullingTime="2.304085323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:40:55.701668716 +0000 UTC m=+370.095472129" watchObservedRunningTime="2026-07-09 02:40:55.702812291 +0000 UTC m=+370.096615727"
Jul 09 02:40:55.705623 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:55.705603    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bszcw\" (UniqueName: \"kubernetes.io/projected/df551de3-1753-4995-91f3-8198f0071445-kube-api-access-bszcw\") pod \"dns-operator-controller-manager-864dd78c4c-77k6w\" (UID: \"df551de3-1753-4995-91f3-8198f0071445\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77k6w"
Jul 09 02:40:55.719878 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:55.719843    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bszcw\" (UniqueName: \"kubernetes.io/projected/df551de3-1753-4995-91f3-8198f0071445-kube-api-access-bszcw\") pod \"dns-operator-controller-manager-864dd78c4c-77k6w\" (UID: \"df551de3-1753-4995-91f3-8198f0071445\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77k6w"
Jul 09 02:40:55.808503 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:55.808427    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77k6w"
Jul 09 02:40:55.938184 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:55.938158    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-77k6w"]
Jul 09 02:40:55.940676 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:40:55.940653    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf551de3_1753_4995_91f3_8198f0071445.slice/crio-9df232d66b6ecb06bf57145a8e6922fc77cb73bd0b85198a91791d191ec3d2b6 WatchSource:0}: Error finding container 9df232d66b6ecb06bf57145a8e6922fc77cb73bd0b85198a91791d191ec3d2b6: Status 404 returned error can't find the container with id 9df232d66b6ecb06bf57145a8e6922fc77cb73bd0b85198a91791d191ec3d2b6
Jul 09 02:40:56.690490 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:56.690455    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77k6w" event={"ID":"df551de3-1753-4995-91f3-8198f0071445","Type":"ContainerStarted","Data":"9df232d66b6ecb06bf57145a8e6922fc77cb73bd0b85198a91791d191ec3d2b6"}
Jul 09 02:40:58.701906 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:58.701870    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77k6w" event={"ID":"df551de3-1753-4995-91f3-8198f0071445","Type":"ContainerStarted","Data":"5b1930f06a152c3b56aaaeb2248c5a9432f430f706353c36db008c626bfb3c0b"}
Jul 09 02:40:58.702418 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:58.701985    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77k6w"
Jul 09 02:40:58.720761 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:58.720688    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77k6w" podStartSLOduration=1.518260342 podStartE2EDuration="3.720671127s" totalImagesPullingTime="2.202410785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:40:58.719694171 +0000 UTC m=+373.113497584" watchObservedRunningTime="2026-07-09 02:40:58.720671127 +0000 UTC m=+373.114474541"
Jul 09 02:40:59.087425 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:59.087346    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"]
Jul 09 02:40:59.090252 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:59.090231    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"
Jul 09 02:40:59.092078 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:59.092060    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-66l5n\""
Jul 09 02:40:59.099350 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:59.099307    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"]
Jul 09 02:40:59.135136 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:59.135099    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkt92\" (UniqueName: \"kubernetes.io/projected/2c072fa2-1610-419c-ac70-d715a637561b-kube-api-access-nkt92\") pod \"limitador-operator-controller-manager-859d7b78bd-bjqlb\" (UID: \"2c072fa2-1610-419c-ac70-d715a637561b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"
Jul 09 02:40:59.236145 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:59.236111    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkt92\" (UniqueName: \"kubernetes.io/projected/2c072fa2-1610-419c-ac70-d715a637561b-kube-api-access-nkt92\") pod \"limitador-operator-controller-manager-859d7b78bd-bjqlb\" (UID: \"2c072fa2-1610-419c-ac70-d715a637561b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"
Jul 09 02:40:59.246590 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:59.246560    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkt92\" (UniqueName: \"kubernetes.io/projected/2c072fa2-1610-419c-ac70-d715a637561b-kube-api-access-nkt92\") pod \"limitador-operator-controller-manager-859d7b78bd-bjqlb\" (UID: \"2c072fa2-1610-419c-ac70-d715a637561b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"
Jul 09 02:40:59.400471 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:59.400445    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"
Jul 09 02:40:59.533387 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:59.533361    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"]
Jul 09 02:40:59.537498 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:40:59.537470    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c072fa2_1610_419c_ac70_d715a637561b.slice/crio-2b52bc1fd06dd24900b872eece74db3dcf3915bccab265f2501baec7a59255a4 WatchSource:0}: Error finding container 2b52bc1fd06dd24900b872eece74db3dcf3915bccab265f2501baec7a59255a4: Status 404 returned error can't find the container with id 2b52bc1fd06dd24900b872eece74db3dcf3915bccab265f2501baec7a59255a4
Jul 09 02:40:59.710409 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:40:59.710252    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb" event={"ID":"2c072fa2-1610-419c-ac70-d715a637561b","Type":"ContainerStarted","Data":"2b52bc1fd06dd24900b872eece74db3dcf3915bccab265f2501baec7a59255a4"}
Jul 09 02:41:01.722546 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:01.722505    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb" event={"ID":"2c072fa2-1610-419c-ac70-d715a637561b","Type":"ContainerStarted","Data":"b06983b0fef02265fa63abc280a2016d397e7e92f2ea313f442a86f4207b16d3"}
Jul 09 02:41:01.722926 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:01.722625    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"
Jul 09 02:41:01.739669 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:01.739617    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb" podStartSLOduration=1.041308735 podStartE2EDuration="2.739603016s" totalImagesPullingTime="1.698294281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:41:01.738938591 +0000 UTC m=+376.132742004" watchObservedRunningTime="2026-07-09 02:41:01.739603016 +0000 UTC m=+376.133406429"
Jul 09 02:41:03.702986 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:03.702951    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"]
Jul 09 02:41:03.706496 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:03.706478    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:03.709023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:03.708998    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2cq5r\""
Jul 09 02:41:03.719884 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:03.719859    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"]
Jul 09 02:41:03.772782 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:03.772755    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p8bq\" (UniqueName: \"kubernetes.io/projected/3aa6f997-3668-4280-a290-f3d5ca909976-kube-api-access-7p8bq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\" (UID: \"3aa6f997-3668-4280-a290-f3d5ca909976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:03.772929 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:03.772788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3aa6f997-3668-4280-a290-f3d5ca909976-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\" (UID: \"3aa6f997-3668-4280-a290-f3d5ca909976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:03.873210 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:03.873169    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p8bq\" (UniqueName: \"kubernetes.io/projected/3aa6f997-3668-4280-a290-f3d5ca909976-kube-api-access-7p8bq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\" (UID: \"3aa6f997-3668-4280-a290-f3d5ca909976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:03.873210 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:03.873213    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3aa6f997-3668-4280-a290-f3d5ca909976-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\" (UID: \"3aa6f997-3668-4280-a290-f3d5ca909976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:03.873603 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:03.873586    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3aa6f997-3668-4280-a290-f3d5ca909976-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\" (UID: \"3aa6f997-3668-4280-a290-f3d5ca909976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:03.884970 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:03.884944    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p8bq\" (UniqueName: \"kubernetes.io/projected/3aa6f997-3668-4280-a290-f3d5ca909976-kube-api-access-7p8bq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\" (UID: \"3aa6f997-3668-4280-a290-f3d5ca909976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:04.016463 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:04.016375    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:04.168465 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:04.168442    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"]
Jul 09 02:41:04.170972 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:41:04.170946    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aa6f997_3668_4280_a290_f3d5ca909976.slice/crio-7842936a0cca9c7d2019e4900e1bc84d35b69f23f592b85ecf2afb31750eb1db WatchSource:0}: Error finding container 7842936a0cca9c7d2019e4900e1bc84d35b69f23f592b85ecf2afb31750eb1db: Status 404 returned error can't find the container with id 7842936a0cca9c7d2019e4900e1bc84d35b69f23f592b85ecf2afb31750eb1db
Jul 09 02:41:04.744581 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:04.744545    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx" event={"ID":"3aa6f997-3668-4280-a290-f3d5ca909976","Type":"ContainerStarted","Data":"7842936a0cca9c7d2019e4900e1bc84d35b69f23f592b85ecf2afb31750eb1db"}
Jul 09 02:41:06.693337 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:06.693286    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jkb7l"
Jul 09 02:41:09.712814 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:09.712782    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77k6w"
Jul 09 02:41:10.779109 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:10.779068    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx" event={"ID":"3aa6f997-3668-4280-a290-f3d5ca909976","Type":"ContainerStarted","Data":"4482bc407833337d81aa5af7c49249563369b14c690f76e64b07be167d8f0272"}
Jul 09 02:41:10.779512 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:10.779152    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:10.803347 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:10.803277    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx" podStartSLOduration=1.679247219 podStartE2EDuration="7.803260791s" totalImagesPullingTime="6.124013572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:41:10.801477226 +0000 UTC m=+385.195280658" watchObservedRunningTime="2026-07-09 02:41:10.803260791 +0000 UTC m=+385.197064204"
Jul 09 02:41:12.739548 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:12.739516    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"
Jul 09 02:41:21.785105 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:21.785076    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:22.895537 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:22.895501    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"]
Jul 09 02:41:22.899187 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:22.899171    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"
Jul 09 02:41:22.910869 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:22.910844    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"]
Jul 09 02:41:22.935411 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:22.935390    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrx57\" (UniqueName: \"kubernetes.io/projected/eb288a42-19bc-4e20-9146-393065267112-kube-api-access-lrx57\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lt4cm\" (UID: \"eb288a42-19bc-4e20-9146-393065267112\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"
Jul 09 02:41:22.935525 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:22.935442    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb288a42-19bc-4e20-9146-393065267112-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lt4cm\" (UID: \"eb288a42-19bc-4e20-9146-393065267112\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"
Jul 09 02:41:23.036297 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.036271    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb288a42-19bc-4e20-9146-393065267112-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lt4cm\" (UID: \"eb288a42-19bc-4e20-9146-393065267112\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"
Jul 09 02:41:23.036431 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.036358    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrx57\" (UniqueName: \"kubernetes.io/projected/eb288a42-19bc-4e20-9146-393065267112-kube-api-access-lrx57\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lt4cm\" (UID: \"eb288a42-19bc-4e20-9146-393065267112\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"
Jul 09 02:41:23.036622 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.036603    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb288a42-19bc-4e20-9146-393065267112-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lt4cm\" (UID: \"eb288a42-19bc-4e20-9146-393065267112\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"
Jul 09 02:41:23.044476 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.044448    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrx57\" (UniqueName: \"kubernetes.io/projected/eb288a42-19bc-4e20-9146-393065267112-kube-api-access-lrx57\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lt4cm\" (UID: \"eb288a42-19bc-4e20-9146-393065267112\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"
Jul 09 02:41:23.208936 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.208878    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"
Jul 09 02:41:23.343042 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.343018    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"]
Jul 09 02:41:23.345238 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:41:23.345213    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb288a42_19bc_4e20_9146_393065267112.slice/crio-32d474640b0bd1deef19dee2050961f11ec82062a65b239e459f823639897e24 WatchSource:0}: Error finding container 32d474640b0bd1deef19dee2050961f11ec82062a65b239e459f823639897e24: Status 404 returned error can't find the container with id 32d474640b0bd1deef19dee2050961f11ec82062a65b239e459f823639897e24
Jul 09 02:41:23.492283 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.492203    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"]
Jul 09 02:41:23.498168 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.498144    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"]
Jul 09 02:41:23.511247 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.511225    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"]
Jul 09 02:41:23.511433 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.511411    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx" podUID="3aa6f997-3668-4280-a290-f3d5ca909976" containerName="manager" containerID="cri-o://4482bc407833337d81aa5af7c49249563369b14c690f76e64b07be167d8f0272" gracePeriod=2
Jul 09 02:41:23.514733 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.514711    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"]
Jul 09 02:41:23.515161 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.515143    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb288a42-19bc-4e20-9146-393065267112" containerName="manager"
Jul 09 02:41:23.515247 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.515165    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb288a42-19bc-4e20-9146-393065267112" containerName="manager"
Jul 09 02:41:23.515302 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.515248    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb288a42-19bc-4e20-9146-393065267112" containerName="manager"
Jul 09 02:41:23.518840 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.518788    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"
Jul 09 02:41:23.523184 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.523157    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"]
Jul 09 02:41:23.524016 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.523930    2578 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"3aa6f997-3668-4280-a290-f3d5ca909976\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\": pods \"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:23.525692 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.525660    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="3aa6f997-3668-4280-a290-f3d5ca909976" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:23.531150 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.531120    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"]
Jul 09 02:41:23.542119 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.542095    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"]
Jul 09 02:41:23.546191 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.543304    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aa6f997-3668-4280-a290-f3d5ca909976" containerName="manager"
Jul 09 02:41:23.546191 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.543345    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aa6f997-3668-4280-a290-f3d5ca909976" containerName="manager"
Jul 09 02:41:23.546191 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.543540    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aa6f997-3668-4280-a290-f3d5ca909976" containerName="manager"
Jul 09 02:41:23.550949 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.550921    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"]
Jul 09 02:41:23.552525 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.551187    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb" podUID="2c072fa2-1610-419c-ac70-d715a637561b" containerName="manager" containerID="cri-o://b06983b0fef02265fa63abc280a2016d397e7e92f2ea313f442a86f4207b16d3" gracePeriod=2
Jul 09 02:41:23.552525 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.552402    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"
Jul 09 02:41:23.556848 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.556822    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"]
Jul 09 02:41:23.559024 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.558996    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="3aa6f997-3668-4280-a290-f3d5ca909976" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:23.559998 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.559976    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"]
Jul 09 02:41:23.560891 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.560866    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="3aa6f997-3668-4280-a290-f3d5ca909976" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:23.571963 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.571943    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lhqz8"]
Jul 09 02:41:23.572279 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.572264    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c072fa2-1610-419c-ac70-d715a637561b" containerName="manager"
Jul 09 02:41:23.572279 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.572278    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c072fa2-1610-419c-ac70-d715a637561b" containerName="manager"
Jul 09 02:41:23.572452 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.572380    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c072fa2-1610-419c-ac70-d715a637561b" containerName="manager"
Jul 09 02:41:23.575330 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.575301    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lhqz8"
Jul 09 02:41:23.584600 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.584577    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lhqz8"]
Jul 09 02:41:23.586501 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.586468    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bjqlb\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="2c072fa2-1610-419c-ac70-d715a637561b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"
Jul 09 02:41:23.588350 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.588307    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="3aa6f997-3668-4280-a290-f3d5ca909976" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:23.606261 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.606232    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bjqlb\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="2c072fa2-1610-419c-ac70-d715a637561b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"
Jul 09 02:41:23.642705 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.642682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjk2r\" (UniqueName: \"kubernetes.io/projected/f4ebfaeb-0956-4f01-8f42-c4c5cf21e52d-kube-api-access-qjk2r\") pod \"limitador-operator-controller-manager-859d7b78bd-lhqz8\" (UID: \"f4ebfaeb-0956-4f01-8f42-c4c5cf21e52d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lhqz8"
Jul 09 02:41:23.642812 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.642729    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f609-17ac-4369-8809-4377f77a201b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wp7rs\" (UID: \"b2b7f609-17ac-4369-8809-4377f77a201b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"
Jul 09 02:41:23.642812 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.642757    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbz8j\" (UniqueName: \"kubernetes.io/projected/e0f0768b-f869-4b7c-933c-025a591c683c-kube-api-access-qbz8j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g5pxd\" (UID: \"e0f0768b-f869-4b7c-933c-025a591c683c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"
Jul 09 02:41:23.642923 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.642831    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhbt5\" (UniqueName: \"kubernetes.io/projected/b2b7f609-17ac-4369-8809-4377f77a201b-kube-api-access-mhbt5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wp7rs\" (UID: \"b2b7f609-17ac-4369-8809-4377f77a201b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"
Jul 09 02:41:23.642923 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.642868    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0f0768b-f869-4b7c-933c-025a591c683c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g5pxd\" (UID: \"e0f0768b-f869-4b7c-933c-025a591c683c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"
Jul 09 02:41:23.743989 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.743925    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhbt5\" (UniqueName: \"kubernetes.io/projected/b2b7f609-17ac-4369-8809-4377f77a201b-kube-api-access-mhbt5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wp7rs\" (UID: \"b2b7f609-17ac-4369-8809-4377f77a201b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"
Jul 09 02:41:23.744080 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.743991    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0f0768b-f869-4b7c-933c-025a591c683c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g5pxd\" (UID: \"e0f0768b-f869-4b7c-933c-025a591c683c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"
Jul 09 02:41:23.744127 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.744085    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjk2r\" (UniqueName: \"kubernetes.io/projected/f4ebfaeb-0956-4f01-8f42-c4c5cf21e52d-kube-api-access-qjk2r\") pod \"limitador-operator-controller-manager-859d7b78bd-lhqz8\" (UID: \"f4ebfaeb-0956-4f01-8f42-c4c5cf21e52d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lhqz8"
Jul 09 02:41:23.744176 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.744132    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f609-17ac-4369-8809-4377f77a201b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wp7rs\" (UID: \"b2b7f609-17ac-4369-8809-4377f77a201b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"
Jul 09 02:41:23.744176 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.744161    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbz8j\" (UniqueName: \"kubernetes.io/projected/e0f0768b-f869-4b7c-933c-025a591c683c-kube-api-access-qbz8j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g5pxd\" (UID: \"e0f0768b-f869-4b7c-933c-025a591c683c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"
Jul 09 02:41:23.744460 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.744435    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0f0768b-f869-4b7c-933c-025a591c683c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g5pxd\" (UID: \"e0f0768b-f869-4b7c-933c-025a591c683c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"
Jul 09 02:41:23.744589 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.744574    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f609-17ac-4369-8809-4377f77a201b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wp7rs\" (UID: \"b2b7f609-17ac-4369-8809-4377f77a201b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"
Jul 09 02:41:23.759259 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.759232    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhbt5\" (UniqueName: \"kubernetes.io/projected/b2b7f609-17ac-4369-8809-4377f77a201b-kube-api-access-mhbt5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wp7rs\" (UID: \"b2b7f609-17ac-4369-8809-4377f77a201b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"
Jul 09 02:41:23.759747 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.759730    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjk2r\" (UniqueName: \"kubernetes.io/projected/f4ebfaeb-0956-4f01-8f42-c4c5cf21e52d-kube-api-access-qjk2r\") pod \"limitador-operator-controller-manager-859d7b78bd-lhqz8\" (UID: \"f4ebfaeb-0956-4f01-8f42-c4c5cf21e52d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lhqz8"
Jul 09 02:41:23.759840 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.759824    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbz8j\" (UniqueName: \"kubernetes.io/projected/e0f0768b-f869-4b7c-933c-025a591c683c-kube-api-access-qbz8j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g5pxd\" (UID: \"e0f0768b-f869-4b7c-933c-025a591c683c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"
Jul 09 02:41:23.783563 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.783539    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:23.785273 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.785245    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bjqlb\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="2c072fa2-1610-419c-ac70-d715a637561b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"
Jul 09 02:41:23.786640 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.786614    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="3aa6f997-3668-4280-a290-f3d5ca909976" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:23.789595 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.789577    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"
Jul 09 02:41:23.791047 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.791021    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="3aa6f997-3668-4280-a290-f3d5ca909976" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:23.792472 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.792450    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bjqlb\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="2c072fa2-1610-419c-ac70-d715a637561b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"
Jul 09 02:41:23.835907 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.835884    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"
Jul 09 02:41:23.842043 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.842018    2578 generic.go:356] "Generic (PLEG): container finished" podID="3aa6f997-3668-4280-a290-f3d5ca909976" containerID="4482bc407833337d81aa5af7c49249563369b14c690f76e64b07be167d8f0272" exitCode=0
Jul 09 02:41:23.842133 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.842053    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:23.842133 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.842117    2578 scope.go:117] "RemoveContainer" containerID="4482bc407833337d81aa5af7c49249563369b14c690f76e64b07be167d8f0272"
Jul 09 02:41:23.843731 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.843709    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="3aa6f997-3668-4280-a290-f3d5ca909976" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:23.844102 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.844082    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm" container="manager"
Jul 09 02:41:23.844508 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.844486    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3aa6f997-3668-4280-a290-f3d5ca909976-extensions-socket-volume\") pod \"3aa6f997-3668-4280-a290-f3d5ca909976\" (UID: \"3aa6f997-3668-4280-a290-f3d5ca909976\") "
Jul 09 02:41:23.844605 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.844530    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7p8bq\" (UniqueName: \"kubernetes.io/projected/3aa6f997-3668-4280-a290-f3d5ca909976-kube-api-access-7p8bq\") pod \"3aa6f997-3668-4280-a290-f3d5ca909976\" (UID: \"3aa6f997-3668-4280-a290-f3d5ca909976\") "
Jul 09 02:41:23.844660 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.844625    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkt92\" (UniqueName: \"kubernetes.io/projected/2c072fa2-1610-419c-ac70-d715a637561b-kube-api-access-nkt92\") pod \"2c072fa2-1610-419c-ac70-d715a637561b\" (UID: \"2c072fa2-1610-419c-ac70-d715a637561b\") "
Jul 09 02:41:23.845103 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.845076    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aa6f997-3668-4280-a290-f3d5ca909976-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3aa6f997-3668-4280-a290-f3d5ca909976" (UID: "3aa6f997-3668-4280-a290-f3d5ca909976"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 02:41:23.845397 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.845367    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bjqlb\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="2c072fa2-1610-419c-ac70-d715a637561b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"
Jul 09 02:41:23.845546 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.845517    2578 generic.go:356] "Generic (PLEG): container finished" podID="2c072fa2-1610-419c-ac70-d715a637561b" containerID="b06983b0fef02265fa63abc280a2016d397e7e92f2ea313f442a86f4207b16d3" exitCode=0
Jul 09 02:41:23.845614 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.845565    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"
Jul 09 02:41:23.846758 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.846730    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bjqlb\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="2c072fa2-1610-419c-ac70-d715a637561b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"
Jul 09 02:41:23.846940 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.846894    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aa6f997-3668-4280-a290-f3d5ca909976-kube-api-access-7p8bq" (OuterVolumeSpecName: "kube-api-access-7p8bq") pod "3aa6f997-3668-4280-a290-f3d5ca909976" (UID: "3aa6f997-3668-4280-a290-f3d5ca909976"). InnerVolumeSpecName "kube-api-access-7p8bq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 02:41:23.847209 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.847191    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c072fa2-1610-419c-ac70-d715a637561b-kube-api-access-nkt92" (OuterVolumeSpecName: "kube-api-access-nkt92") pod "2c072fa2-1610-419c-ac70-d715a637561b" (UID: "2c072fa2-1610-419c-ac70-d715a637561b"). InnerVolumeSpecName "kube-api-access-nkt92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 02:41:23.848103 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.848078    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="3aa6f997-3668-4280-a290-f3d5ca909976" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:23.849462 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.849435    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lt4cm\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="eb288a42-19bc-4e20-9146-393065267112" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"
Jul 09 02:41:23.850761 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.850730    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ks9rx\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="3aa6f997-3668-4280-a290-f3d5ca909976" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ks9rx"
Jul 09 02:41:23.852286 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.852141    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lt4cm\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="eb288a42-19bc-4e20-9146-393065267112" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"
Jul 09 02:41:23.852286 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.852189    2578 scope.go:117] "RemoveContainer" containerID="4482bc407833337d81aa5af7c49249563369b14c690f76e64b07be167d8f0272"
Jul 09 02:41:23.852498 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.852478    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4482bc407833337d81aa5af7c49249563369b14c690f76e64b07be167d8f0272\": container with ID starting with 4482bc407833337d81aa5af7c49249563369b14c690f76e64b07be167d8f0272 not found: ID does not exist" containerID="4482bc407833337d81aa5af7c49249563369b14c690f76e64b07be167d8f0272"
Jul 09 02:41:23.852569 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.852509    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4482bc407833337d81aa5af7c49249563369b14c690f76e64b07be167d8f0272"} err="failed to get container status \"4482bc407833337d81aa5af7c49249563369b14c690f76e64b07be167d8f0272\": rpc error: code = NotFound desc = could not find container \"4482bc407833337d81aa5af7c49249563369b14c690f76e64b07be167d8f0272\": container with ID starting with 4482bc407833337d81aa5af7c49249563369b14c690f76e64b07be167d8f0272 not found: ID does not exist"
Jul 09 02:41:23.852569 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.852532    2578 scope.go:117] "RemoveContainer" containerID="b06983b0fef02265fa63abc280a2016d397e7e92f2ea313f442a86f4207b16d3"
Jul 09 02:41:23.853621 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.853599    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bjqlb\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="2c072fa2-1610-419c-ac70-d715a637561b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bjqlb"
Jul 09 02:41:23.860533 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.860505    2578 scope.go:117] "RemoveContainer" containerID="b06983b0fef02265fa63abc280a2016d397e7e92f2ea313f442a86f4207b16d3"
Jul 09 02:41:23.860836 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:23.860818    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b06983b0fef02265fa63abc280a2016d397e7e92f2ea313f442a86f4207b16d3\": container with ID starting with b06983b0fef02265fa63abc280a2016d397e7e92f2ea313f442a86f4207b16d3 not found: ID does not exist" containerID="b06983b0fef02265fa63abc280a2016d397e7e92f2ea313f442a86f4207b16d3"
Jul 09 02:41:23.860901 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.860846    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b06983b0fef02265fa63abc280a2016d397e7e92f2ea313f442a86f4207b16d3"} err="failed to get container status \"b06983b0fef02265fa63abc280a2016d397e7e92f2ea313f442a86f4207b16d3\": rpc error: code = NotFound desc = could not find container \"b06983b0fef02265fa63abc280a2016d397e7e92f2ea313f442a86f4207b16d3\": container with ID starting with b06983b0fef02265fa63abc280a2016d397e7e92f2ea313f442a86f4207b16d3 not found: ID does not exist"
Jul 09 02:41:23.935502 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.935472    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"
Jul 09 02:41:23.940772 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.940749    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lhqz8"
Jul 09 02:41:23.945934 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.945910    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkt92\" (UniqueName: \"kubernetes.io/projected/2c072fa2-1610-419c-ac70-d715a637561b-kube-api-access-nkt92\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:41:23.946029 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.945935    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3aa6f997-3668-4280-a290-f3d5ca909976-extensions-socket-volume\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:41:23.946029 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.945949    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7p8bq\" (UniqueName: \"kubernetes.io/projected/3aa6f997-3668-4280-a290-f3d5ca909976-kube-api-access-7p8bq\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:41:23.975386 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:23.975360    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"]
Jul 09 02:41:23.978005 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:41:23.977974    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2b7f609_17ac_4369_8809_4377f77a201b.slice/crio-587d030e962cd8b00a6dd071fc84291fca1da963035abe0775d4c7c9f78eca37 WatchSource:0}: Error finding container 587d030e962cd8b00a6dd071fc84291fca1da963035abe0775d4c7c9f78eca37: Status 404 returned error can't find the container with id 587d030e962cd8b00a6dd071fc84291fca1da963035abe0775d4c7c9f78eca37
Jul 09 02:41:24.094625 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.094597    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"]
Jul 09 02:41:24.097122 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:41:24.097097    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0f0768b_f869_4b7c_933c_025a591c683c.slice/crio-c0e063b2926a286b130cb3d0b26c0d2a57b6704a5479185dcd4199020b9efd0d WatchSource:0}: Error finding container c0e063b2926a286b130cb3d0b26c0d2a57b6704a5479185dcd4199020b9efd0d: Status 404 returned error can't find the container with id c0e063b2926a286b130cb3d0b26c0d2a57b6704a5479185dcd4199020b9efd0d
Jul 09 02:41:24.104004 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.103947    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lhqz8"]
Jul 09 02:41:24.106148 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:41:24.106120    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4ebfaeb_0956_4f01_8f42_c4c5cf21e52d.slice/crio-ce495d1c9141a9dffa30cddc73d3ba837ada4192dc213b3fa7222078b8ce927e WatchSource:0}: Error finding container ce495d1c9141a9dffa30cddc73d3ba837ada4192dc213b3fa7222078b8ce927e: Status 404 returned error can't find the container with id ce495d1c9141a9dffa30cddc73d3ba837ada4192dc213b3fa7222078b8ce927e
Jul 09 02:41:24.118157 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.118134    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c072fa2-1610-419c-ac70-d715a637561b" path="/var/lib/kubelet/pods/2c072fa2-1610-419c-ac70-d715a637561b/volumes"
Jul 09 02:41:24.118529 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.118514    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3aa6f997-3668-4280-a290-f3d5ca909976" path="/var/lib/kubelet/pods/3aa6f997-3668-4280-a290-f3d5ca909976/volumes"
Jul 09 02:41:24.852924 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.852888    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs" event={"ID":"b2b7f609-17ac-4369-8809-4377f77a201b","Type":"ContainerStarted","Data":"64c29e1999bdf478a6d00c88141f2d6c28c9f5e5807b233a0efb2fa5601b3076"}
Jul 09 02:41:24.853111 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.852938    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs" event={"ID":"b2b7f609-17ac-4369-8809-4377f77a201b","Type":"ContainerStarted","Data":"587d030e962cd8b00a6dd071fc84291fca1da963035abe0775d4c7c9f78eca37"}
Jul 09 02:41:24.853111 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.852990    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"
Jul 09 02:41:24.854598 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.854562    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm" podUID="eb288a42-19bc-4e20-9146-393065267112" containerName="manager" containerID="cri-o://4c140030d837bedbd28e142bc2d65354dee4d01500666a8f5b21fedd59f11c25" gracePeriod=2
Jul 09 02:41:24.856197 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.856173    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lhqz8" event={"ID":"f4ebfaeb-0956-4f01-8f42-c4c5cf21e52d","Type":"ContainerStarted","Data":"bb33338f6a7ef20d71508469e94f67493322404c496c2be79bcb7b33aab2a12a"}
Jul 09 02:41:24.856362 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.856229    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lhqz8" event={"ID":"f4ebfaeb-0956-4f01-8f42-c4c5cf21e52d","Type":"ContainerStarted","Data":"ce495d1c9141a9dffa30cddc73d3ba837ada4192dc213b3fa7222078b8ce927e"}
Jul 09 02:41:24.856362 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.856256    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lhqz8"
Jul 09 02:41:24.859127 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.858816    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd" event={"ID":"e0f0768b-f869-4b7c-933c-025a591c683c","Type":"ContainerStarted","Data":"a824198a7c44444f2c5e94318c22d300dd90b03ae17190bb960614def88d35a0"}
Jul 09 02:41:24.859127 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.858847    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd" event={"ID":"e0f0768b-f869-4b7c-933c-025a591c683c","Type":"ContainerStarted","Data":"c0e063b2926a286b130cb3d0b26c0d2a57b6704a5479185dcd4199020b9efd0d"}
Jul 09 02:41:24.859127 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.858941    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"
Jul 09 02:41:24.870334 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.870276    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs" podStartSLOduration=1.870263636 podStartE2EDuration="1.870263636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:41:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:41:24.868571292 +0000 UTC m=+399.262374706" watchObservedRunningTime="2026-07-09 02:41:24.870263636 +0000 UTC m=+399.264067104"
Jul 09 02:41:24.889850 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.889808    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd" podStartSLOduration=1.889794336 podStartE2EDuration="1.889794336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:41:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:41:24.889515771 +0000 UTC m=+399.283319208" watchObservedRunningTime="2026-07-09 02:41:24.889794336 +0000 UTC m=+399.283597748"
Jul 09 02:41:24.909069 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:24.909033    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lhqz8" podStartSLOduration=1.90902154 podStartE2EDuration="1.90902154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:41:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:41:24.908031315 +0000 UTC m=+399.301834730" watchObservedRunningTime="2026-07-09 02:41:24.90902154 +0000 UTC m=+399.302824953"
Jul 09 02:41:25.092892 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:25.092865    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"
Jul 09 02:41:25.095186 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:25.095163    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lt4cm\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="eb288a42-19bc-4e20-9146-393065267112" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"
Jul 09 02:41:25.153482 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:25.153410    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb288a42-19bc-4e20-9146-393065267112-extensions-socket-volume\") pod \"eb288a42-19bc-4e20-9146-393065267112\" (UID: \"eb288a42-19bc-4e20-9146-393065267112\") "
Jul 09 02:41:25.153594 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:25.153491    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrx57\" (UniqueName: \"kubernetes.io/projected/eb288a42-19bc-4e20-9146-393065267112-kube-api-access-lrx57\") pod \"eb288a42-19bc-4e20-9146-393065267112\" (UID: \"eb288a42-19bc-4e20-9146-393065267112\") "
Jul 09 02:41:25.153688 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:25.153661    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb288a42-19bc-4e20-9146-393065267112-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "eb288a42-19bc-4e20-9146-393065267112" (UID: "eb288a42-19bc-4e20-9146-393065267112"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 02:41:25.155462 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:25.155443    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb288a42-19bc-4e20-9146-393065267112-kube-api-access-lrx57" (OuterVolumeSpecName: "kube-api-access-lrx57") pod "eb288a42-19bc-4e20-9146-393065267112" (UID: "eb288a42-19bc-4e20-9146-393065267112"). InnerVolumeSpecName "kube-api-access-lrx57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 02:41:25.254161 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:25.254133    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb288a42-19bc-4e20-9146-393065267112-extensions-socket-volume\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:41:25.254161 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:25.254157    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrx57\" (UniqueName: \"kubernetes.io/projected/eb288a42-19bc-4e20-9146-393065267112-kube-api-access-lrx57\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:41:25.865040 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:25.865002    2578 generic.go:356] "Generic (PLEG): container finished" podID="eb288a42-19bc-4e20-9146-393065267112" containerID="4c140030d837bedbd28e142bc2d65354dee4d01500666a8f5b21fedd59f11c25" exitCode=2
Jul 09 02:41:25.865228 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:25.865060    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"
Jul 09 02:41:25.865228 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:25.865094    2578 scope.go:117] "RemoveContainer" containerID="4c140030d837bedbd28e142bc2d65354dee4d01500666a8f5b21fedd59f11c25"
Jul 09 02:41:25.866778 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:25.866747    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lt4cm\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="eb288a42-19bc-4e20-9146-393065267112" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"
Jul 09 02:41:25.873999 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:25.873983    2578 scope.go:117] "RemoveContainer" containerID="4c140030d837bedbd28e142bc2d65354dee4d01500666a8f5b21fedd59f11c25"
Jul 09 02:41:25.874267 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:25.874250    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c140030d837bedbd28e142bc2d65354dee4d01500666a8f5b21fedd59f11c25\": container with ID starting with 4c140030d837bedbd28e142bc2d65354dee4d01500666a8f5b21fedd59f11c25 not found: ID does not exist" containerID="4c140030d837bedbd28e142bc2d65354dee4d01500666a8f5b21fedd59f11c25"
Jul 09 02:41:25.874355 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:25.874280    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c140030d837bedbd28e142bc2d65354dee4d01500666a8f5b21fedd59f11c25"} err="failed to get container status \"4c140030d837bedbd28e142bc2d65354dee4d01500666a8f5b21fedd59f11c25\": rpc error: code = NotFound desc = could not find container \"4c140030d837bedbd28e142bc2d65354dee4d01500666a8f5b21fedd59f11c25\": container with ID starting with 4c140030d837bedbd28e142bc2d65354dee4d01500666a8f5b21fedd59f11c25 not found: ID does not exist"
Jul 09 02:41:25.874953 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:25.874931    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lt4cm\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="eb288a42-19bc-4e20-9146-393065267112" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"
Jul 09 02:41:26.115179 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:26.115095    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lt4cm\" is forbidden: User \"system:node:ip-10-0-129-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-236.ec2.internal' and this object" podUID="eb288a42-19bc-4e20-9146-393065267112" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lt4cm"
Jul 09 02:41:26.117199 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:26.117178    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb288a42-19bc-4e20-9146-393065267112" path="/var/lib/kubelet/pods/eb288a42-19bc-4e20-9146-393065267112/volumes"
Jul 09 02:41:35.868156 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:35.868128    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"
Jul 09 02:41:35.868572 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:35.868198    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"
Jul 09 02:41:35.868572 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:35.868247    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lhqz8"
Jul 09 02:41:35.938381 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:35.938352    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"]
Jul 09 02:41:35.938549 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:35.938524    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd" podUID="e0f0768b-f869-4b7c-933c-025a591c683c" containerName="manager" containerID="cri-o://a824198a7c44444f2c5e94318c22d300dd90b03ae17190bb960614def88d35a0" gracePeriod=10
Jul 09 02:41:36.189743 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.189719    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"
Jul 09 02:41:36.209740 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.209704    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"]
Jul 09 02:41:36.210169 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.210150    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0f0768b-f869-4b7c-933c-025a591c683c" containerName="manager"
Jul 09 02:41:36.210169 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.210171    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0f0768b-f869-4b7c-933c-025a591c683c" containerName="manager"
Jul 09 02:41:36.210306 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.210245    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0f0768b-f869-4b7c-933c-025a591c683c" containerName="manager"
Jul 09 02:41:36.213685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.213662    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"
Jul 09 02:41:36.229980 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.229958    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"]
Jul 09 02:41:36.347634 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.347604    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbz8j\" (UniqueName: \"kubernetes.io/projected/e0f0768b-f869-4b7c-933c-025a591c683c-kube-api-access-qbz8j\") pod \"e0f0768b-f869-4b7c-933c-025a591c683c\" (UID: \"e0f0768b-f869-4b7c-933c-025a591c683c\") "
Jul 09 02:41:36.347634 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.347638    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0f0768b-f869-4b7c-933c-025a591c683c-extensions-socket-volume\") pod \"e0f0768b-f869-4b7c-933c-025a591c683c\" (UID: \"e0f0768b-f869-4b7c-933c-025a591c683c\") "
Jul 09 02:41:36.347815 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.347772    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d6dc778-706e-42e2-9f42-b9bff65d93e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-57jfs\" (UID: \"4d6dc778-706e-42e2-9f42-b9bff65d93e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"
Jul 09 02:41:36.347815 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.347811    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf9kj\" (UniqueName: \"kubernetes.io/projected/4d6dc778-706e-42e2-9f42-b9bff65d93e0-kube-api-access-nf9kj\") pod \"kuadrant-operator-controller-manager-86878c769f-57jfs\" (UID: \"4d6dc778-706e-42e2-9f42-b9bff65d93e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"
Jul 09 02:41:36.348059 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.348037    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0f0768b-f869-4b7c-933c-025a591c683c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e0f0768b-f869-4b7c-933c-025a591c683c" (UID: "e0f0768b-f869-4b7c-933c-025a591c683c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 02:41:36.349571 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.349547    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0f0768b-f869-4b7c-933c-025a591c683c-kube-api-access-qbz8j" (OuterVolumeSpecName: "kube-api-access-qbz8j") pod "e0f0768b-f869-4b7c-933c-025a591c683c" (UID: "e0f0768b-f869-4b7c-933c-025a591c683c"). InnerVolumeSpecName "kube-api-access-qbz8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 02:41:36.448469 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.448392    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d6dc778-706e-42e2-9f42-b9bff65d93e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-57jfs\" (UID: \"4d6dc778-706e-42e2-9f42-b9bff65d93e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"
Jul 09 02:41:36.448469 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.448454    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf9kj\" (UniqueName: \"kubernetes.io/projected/4d6dc778-706e-42e2-9f42-b9bff65d93e0-kube-api-access-nf9kj\") pod \"kuadrant-operator-controller-manager-86878c769f-57jfs\" (UID: \"4d6dc778-706e-42e2-9f42-b9bff65d93e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"
Jul 09 02:41:36.448670 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.448519    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbz8j\" (UniqueName: \"kubernetes.io/projected/e0f0768b-f869-4b7c-933c-025a591c683c-kube-api-access-qbz8j\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:41:36.448670 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.448529    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0f0768b-f869-4b7c-933c-025a591c683c-extensions-socket-volume\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:41:36.448849 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.448827    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d6dc778-706e-42e2-9f42-b9bff65d93e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-57jfs\" (UID: \"4d6dc778-706e-42e2-9f42-b9bff65d93e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"
Jul 09 02:41:36.466379 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.466349    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf9kj\" (UniqueName: \"kubernetes.io/projected/4d6dc778-706e-42e2-9f42-b9bff65d93e0-kube-api-access-nf9kj\") pod \"kuadrant-operator-controller-manager-86878c769f-57jfs\" (UID: \"4d6dc778-706e-42e2-9f42-b9bff65d93e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"
Jul 09 02:41:36.524411 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.524375    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"
Jul 09 02:41:36.677327 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.677291    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"]
Jul 09 02:41:36.679305 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:41:36.679279    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d6dc778_706e_42e2_9f42_b9bff65d93e0.slice/crio-a5a5052f96a00ef73568dcdd72817985640ee4c86fe21c9b5174daeca3da5748 WatchSource:0}: Error finding container a5a5052f96a00ef73568dcdd72817985640ee4c86fe21c9b5174daeca3da5748: Status 404 returned error can't find the container with id a5a5052f96a00ef73568dcdd72817985640ee4c86fe21c9b5174daeca3da5748
Jul 09 02:41:36.917562 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.917525    2578 generic.go:356] "Generic (PLEG): container finished" podID="e0f0768b-f869-4b7c-933c-025a591c683c" containerID="a824198a7c44444f2c5e94318c22d300dd90b03ae17190bb960614def88d35a0" exitCode=0
Jul 09 02:41:36.917952 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.917582    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"
Jul 09 02:41:36.917952 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.917602    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd" event={"ID":"e0f0768b-f869-4b7c-933c-025a591c683c","Type":"ContainerDied","Data":"a824198a7c44444f2c5e94318c22d300dd90b03ae17190bb960614def88d35a0"}
Jul 09 02:41:36.917952 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.917657    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd" event={"ID":"e0f0768b-f869-4b7c-933c-025a591c683c","Type":"ContainerDied","Data":"c0e063b2926a286b130cb3d0b26c0d2a57b6704a5479185dcd4199020b9efd0d"}
Jul 09 02:41:36.917952 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.917681    2578 scope.go:117] "RemoveContainer" containerID="a824198a7c44444f2c5e94318c22d300dd90b03ae17190bb960614def88d35a0"
Jul 09 02:41:36.919396 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.919373    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs" event={"ID":"4d6dc778-706e-42e2-9f42-b9bff65d93e0","Type":"ContainerStarted","Data":"91a44ab22a3105368ad418c4a649679b7808046140e8c48319963242268e5a3b"}
Jul 09 02:41:36.919535 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.919425    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"
Jul 09 02:41:36.919535 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.919448    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs" event={"ID":"4d6dc778-706e-42e2-9f42-b9bff65d93e0","Type":"ContainerStarted","Data":"a5a5052f96a00ef73568dcdd72817985640ee4c86fe21c9b5174daeca3da5748"}
Jul 09 02:41:36.926986 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.926966    2578 scope.go:117] "RemoveContainer" containerID="a824198a7c44444f2c5e94318c22d300dd90b03ae17190bb960614def88d35a0"
Jul 09 02:41:36.927281 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:36.927254    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a824198a7c44444f2c5e94318c22d300dd90b03ae17190bb960614def88d35a0\": container with ID starting with a824198a7c44444f2c5e94318c22d300dd90b03ae17190bb960614def88d35a0 not found: ID does not exist" containerID="a824198a7c44444f2c5e94318c22d300dd90b03ae17190bb960614def88d35a0"
Jul 09 02:41:36.927394 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.927280    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a824198a7c44444f2c5e94318c22d300dd90b03ae17190bb960614def88d35a0"} err="failed to get container status \"a824198a7c44444f2c5e94318c22d300dd90b03ae17190bb960614def88d35a0\": rpc error: code = NotFound desc = could not find container \"a824198a7c44444f2c5e94318c22d300dd90b03ae17190bb960614def88d35a0\": container with ID starting with a824198a7c44444f2c5e94318c22d300dd90b03ae17190bb960614def88d35a0 not found: ID does not exist"
Jul 09 02:41:36.942713 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.942664    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs" podStartSLOduration=0.942646548 podStartE2EDuration="942.646548ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:41:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:41:36.94152988 +0000 UTC m=+411.335333293" watchObservedRunningTime="2026-07-09 02:41:36.942646548 +0000 UTC m=+411.336449964"
Jul 09 02:41:36.967260 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.967238    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"]
Jul 09 02:41:36.975808 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:36.975760    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5pxd"]
Jul 09 02:41:38.117748 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:38.117717    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0f0768b-f869-4b7c-933c-025a591c683c" path="/var/lib/kubelet/pods/e0f0768b-f869-4b7c-933c-025a591c683c/volumes"
Jul 09 02:41:47.927963 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:47.927883    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"
Jul 09 02:41:47.971424 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:47.971393    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"]
Jul 09 02:41:47.971616 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:47.971580    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs" podUID="b2b7f609-17ac-4369-8809-4377f77a201b" containerName="manager" containerID="cri-o://64c29e1999bdf478a6d00c88141f2d6c28c9f5e5807b233a0efb2fa5601b3076" gracePeriod=10
Jul 09 02:41:48.211912 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:48.211889    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"
Jul 09 02:41:48.352179 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:48.352151    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f609-17ac-4369-8809-4377f77a201b-extensions-socket-volume\") pod \"b2b7f609-17ac-4369-8809-4377f77a201b\" (UID: \"b2b7f609-17ac-4369-8809-4377f77a201b\") "
Jul 09 02:41:48.352380 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:48.352223    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhbt5\" (UniqueName: \"kubernetes.io/projected/b2b7f609-17ac-4369-8809-4377f77a201b-kube-api-access-mhbt5\") pod \"b2b7f609-17ac-4369-8809-4377f77a201b\" (UID: \"b2b7f609-17ac-4369-8809-4377f77a201b\") "
Jul 09 02:41:48.352552 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:48.352529    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2b7f609-17ac-4369-8809-4377f77a201b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b2b7f609-17ac-4369-8809-4377f77a201b" (UID: "b2b7f609-17ac-4369-8809-4377f77a201b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 02:41:48.354235 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:48.354216    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2b7f609-17ac-4369-8809-4377f77a201b-kube-api-access-mhbt5" (OuterVolumeSpecName: "kube-api-access-mhbt5") pod "b2b7f609-17ac-4369-8809-4377f77a201b" (UID: "b2b7f609-17ac-4369-8809-4377f77a201b"). InnerVolumeSpecName "kube-api-access-mhbt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 02:41:48.453825 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:48.453748    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhbt5\" (UniqueName: \"kubernetes.io/projected/b2b7f609-17ac-4369-8809-4377f77a201b-kube-api-access-mhbt5\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:41:48.453825 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:48.453775    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f609-17ac-4369-8809-4377f77a201b-extensions-socket-volume\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:41:48.977568 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:48.977533    2578 generic.go:356] "Generic (PLEG): container finished" podID="b2b7f609-17ac-4369-8809-4377f77a201b" containerID="64c29e1999bdf478a6d00c88141f2d6c28c9f5e5807b233a0efb2fa5601b3076" exitCode=0
Jul 09 02:41:48.978023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:48.977577    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"
Jul 09 02:41:48.978023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:48.977595    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs" event={"ID":"b2b7f609-17ac-4369-8809-4377f77a201b","Type":"ContainerDied","Data":"64c29e1999bdf478a6d00c88141f2d6c28c9f5e5807b233a0efb2fa5601b3076"}
Jul 09 02:41:48.978023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:48.977635    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs" event={"ID":"b2b7f609-17ac-4369-8809-4377f77a201b","Type":"ContainerDied","Data":"587d030e962cd8b00a6dd071fc84291fca1da963035abe0775d4c7c9f78eca37"}
Jul 09 02:41:48.978023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:48.977655    2578 scope.go:117] "RemoveContainer" containerID="64c29e1999bdf478a6d00c88141f2d6c28c9f5e5807b233a0efb2fa5601b3076"
Jul 09 02:41:48.986945 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:48.986928    2578 scope.go:117] "RemoveContainer" containerID="64c29e1999bdf478a6d00c88141f2d6c28c9f5e5807b233a0efb2fa5601b3076"
Jul 09 02:41:48.987235 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:41:48.987218    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64c29e1999bdf478a6d00c88141f2d6c28c9f5e5807b233a0efb2fa5601b3076\": container with ID starting with 64c29e1999bdf478a6d00c88141f2d6c28c9f5e5807b233a0efb2fa5601b3076 not found: ID does not exist" containerID="64c29e1999bdf478a6d00c88141f2d6c28c9f5e5807b233a0efb2fa5601b3076"
Jul 09 02:41:48.987293 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:48.987243    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64c29e1999bdf478a6d00c88141f2d6c28c9f5e5807b233a0efb2fa5601b3076"} err="failed to get container status \"64c29e1999bdf478a6d00c88141f2d6c28c9f5e5807b233a0efb2fa5601b3076\": rpc error: code = NotFound desc = could not find container \"64c29e1999bdf478a6d00c88141f2d6c28c9f5e5807b233a0efb2fa5601b3076\": container with ID starting with 64c29e1999bdf478a6d00c88141f2d6c28c9f5e5807b233a0efb2fa5601b3076 not found: ID does not exist"
Jul 09 02:41:49.014727 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:49.014695    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"]
Jul 09 02:41:49.025101 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:49.025080    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wp7rs"]
Jul 09 02:41:50.117629 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:41:50.117596    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2b7f609-17ac-4369-8809-4377f77a201b" path="/var/lib/kubelet/pods/b2b7f609-17ac-4369-8809-4377f77a201b/volumes"
Jul 09 02:42:04.978292 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:04.978259    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z49v2"]
Jul 09 02:42:04.978874 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:04.978739    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2b7f609-17ac-4369-8809-4377f77a201b" containerName="manager"
Jul 09 02:42:04.978874 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:04.978759    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2b7f609-17ac-4369-8809-4377f77a201b" containerName="manager"
Jul 09 02:42:04.978874 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:04.978867    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2b7f609-17ac-4369-8809-4377f77a201b" containerName="manager"
Jul 09 02:42:04.983424 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:04.983403    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2"
Jul 09 02:42:04.985262 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:04.985242    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 09 02:42:04.985380 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:04.985282    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-nbpf4\""
Jul 09 02:42:04.988797 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:04.988772    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z49v2"]
Jul 09 02:42:05.075643 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:05.075612    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z49v2"]
Jul 09 02:42:05.086015 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:05.085986    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7c2a2922-e404-4002-85af-d18fdde2b5fb-config-file\") pod \"limitador-limitador-64c94fd64f-z49v2\" (UID: \"7c2a2922-e404-4002-85af-d18fdde2b5fb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2"
Jul 09 02:42:05.086154 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:05.086109    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf4zz\" (UniqueName: \"kubernetes.io/projected/7c2a2922-e404-4002-85af-d18fdde2b5fb-kube-api-access-bf4zz\") pod \"limitador-limitador-64c94fd64f-z49v2\" (UID: \"7c2a2922-e404-4002-85af-d18fdde2b5fb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2"
Jul 09 02:42:05.187284 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:05.187252    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7c2a2922-e404-4002-85af-d18fdde2b5fb-config-file\") pod \"limitador-limitador-64c94fd64f-z49v2\" (UID: \"7c2a2922-e404-4002-85af-d18fdde2b5fb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2"
Jul 09 02:42:05.187459 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:05.187335    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf4zz\" (UniqueName: \"kubernetes.io/projected/7c2a2922-e404-4002-85af-d18fdde2b5fb-kube-api-access-bf4zz\") pod \"limitador-limitador-64c94fd64f-z49v2\" (UID: \"7c2a2922-e404-4002-85af-d18fdde2b5fb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2"
Jul 09 02:42:05.187871 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:05.187851    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7c2a2922-e404-4002-85af-d18fdde2b5fb-config-file\") pod \"limitador-limitador-64c94fd64f-z49v2\" (UID: \"7c2a2922-e404-4002-85af-d18fdde2b5fb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2"
Jul 09 02:42:05.194260 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:05.194237    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf4zz\" (UniqueName: \"kubernetes.io/projected/7c2a2922-e404-4002-85af-d18fdde2b5fb-kube-api-access-bf4zz\") pod \"limitador-limitador-64c94fd64f-z49v2\" (UID: \"7c2a2922-e404-4002-85af-d18fdde2b5fb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2"
Jul 09 02:42:05.295197 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:05.295130    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2"
Jul 09 02:42:05.428186 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:05.428156    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z49v2"]
Jul 09 02:42:05.431666 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:42:05.431613    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c2a2922_e404_4002_85af_d18fdde2b5fb.slice/crio-6f29d7d8129fae0ca7920eb24e7e861f2c421ee153e88a217e8e20b5a5c8ad86 WatchSource:0}: Error finding container 6f29d7d8129fae0ca7920eb24e7e861f2c421ee153e88a217e8e20b5a5c8ad86: Status 404 returned error can't find the container with id 6f29d7d8129fae0ca7920eb24e7e861f2c421ee153e88a217e8e20b5a5c8ad86
Jul 09 02:42:06.059983 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:06.059946    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2" event={"ID":"7c2a2922-e404-4002-85af-d18fdde2b5fb","Type":"ContainerStarted","Data":"6f29d7d8129fae0ca7920eb24e7e861f2c421ee153e88a217e8e20b5a5c8ad86"}
Jul 09 02:42:09.076539 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:09.076504    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2" event={"ID":"7c2a2922-e404-4002-85af-d18fdde2b5fb","Type":"ContainerStarted","Data":"ea975be90dda6c49147f24e933d8beda3c527525fa6cb477daf39b3e92eb2171"}
Jul 09 02:42:09.076923 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:09.076571    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2"
Jul 09 02:42:09.091618 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:09.091567    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2" podStartSLOduration=2.271596862 podStartE2EDuration="5.091551967s" totalImagesPullingTime="2.819955105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:42:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:42:09.090629726 +0000 UTC m=+443.484433150" watchObservedRunningTime="2026-07-09 02:42:09.091551967 +0000 UTC m=+443.485355385"
Jul 09 02:42:20.082368 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:20.082338    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2"
Jul 09 02:42:22.670232 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:22.670203    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z49v2"]
Jul 09 02:42:22.670601 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:22.670402    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2" podUID="7c2a2922-e404-4002-85af-d18fdde2b5fb" containerName="limitador" containerID="cri-o://ea975be90dda6c49147f24e933d8beda3c527525fa6cb477daf39b3e92eb2171" gracePeriod=30
Jul 09 02:42:23.144303 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:23.144268    2578 generic.go:356] "Generic (PLEG): container finished" podID="7c2a2922-e404-4002-85af-d18fdde2b5fb" containerID="ea975be90dda6c49147f24e933d8beda3c527525fa6cb477daf39b3e92eb2171" exitCode=0
Jul 09 02:42:23.144439 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:23.144355    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2" event={"ID":"7c2a2922-e404-4002-85af-d18fdde2b5fb","Type":"ContainerDied","Data":"ea975be90dda6c49147f24e933d8beda3c527525fa6cb477daf39b3e92eb2171"}
Jul 09 02:42:23.219617 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:23.219592    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2"
Jul 09 02:42:23.348902 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:23.348824    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7c2a2922-e404-4002-85af-d18fdde2b5fb-config-file\") pod \"7c2a2922-e404-4002-85af-d18fdde2b5fb\" (UID: \"7c2a2922-e404-4002-85af-d18fdde2b5fb\") "
Jul 09 02:42:23.348902 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:23.348884    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bf4zz\" (UniqueName: \"kubernetes.io/projected/7c2a2922-e404-4002-85af-d18fdde2b5fb-kube-api-access-bf4zz\") pod \"7c2a2922-e404-4002-85af-d18fdde2b5fb\" (UID: \"7c2a2922-e404-4002-85af-d18fdde2b5fb\") "
Jul 09 02:42:23.349175 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:23.349152    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c2a2922-e404-4002-85af-d18fdde2b5fb-config-file" (OuterVolumeSpecName: "config-file") pod "7c2a2922-e404-4002-85af-d18fdde2b5fb" (UID: "7c2a2922-e404-4002-85af-d18fdde2b5fb"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 02:42:23.350895 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:23.350867    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c2a2922-e404-4002-85af-d18fdde2b5fb-kube-api-access-bf4zz" (OuterVolumeSpecName: "kube-api-access-bf4zz") pod "7c2a2922-e404-4002-85af-d18fdde2b5fb" (UID: "7c2a2922-e404-4002-85af-d18fdde2b5fb"). InnerVolumeSpecName "kube-api-access-bf4zz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 02:42:23.450389 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:23.450357    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bf4zz\" (UniqueName: \"kubernetes.io/projected/7c2a2922-e404-4002-85af-d18fdde2b5fb-kube-api-access-bf4zz\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:42:23.450389 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:23.450383    2578 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7c2a2922-e404-4002-85af-d18fdde2b5fb-config-file\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:42:24.149721 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:24.149701    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2"
Jul 09 02:42:24.150103 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:24.149690    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z49v2" event={"ID":"7c2a2922-e404-4002-85af-d18fdde2b5fb","Type":"ContainerDied","Data":"6f29d7d8129fae0ca7920eb24e7e861f2c421ee153e88a217e8e20b5a5c8ad86"}
Jul 09 02:42:24.150103 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:24.149837    2578 scope.go:117] "RemoveContainer" containerID="ea975be90dda6c49147f24e933d8beda3c527525fa6cb477daf39b3e92eb2171"
Jul 09 02:42:24.165180 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:24.165155    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z49v2"]
Jul 09 02:42:24.168100 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:24.168078    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z49v2"]
Jul 09 02:42:25.891852 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:25.891814    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5q5fj"]
Jul 09 02:42:25.892401 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:25.892378    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c2a2922-e404-4002-85af-d18fdde2b5fb" containerName="limitador"
Jul 09 02:42:25.892556 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:25.892542    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c2a2922-e404-4002-85af-d18fdde2b5fb" containerName="limitador"
Jul 09 02:42:25.892764 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:25.892744    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c2a2922-e404-4002-85af-d18fdde2b5fb" containerName="limitador"
Jul 09 02:42:25.907749 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:25.907725    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5q5fj"]
Jul 09 02:42:25.907852 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:25.907825    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q5fj"
Jul 09 02:42:25.910748 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:25.910725    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 09 02:42:25.910878 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:25.910860    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-7g6j4\""
Jul 09 02:42:25.911253 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:25.911236    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 02:42:25.911388 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:25.911266    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 02:42:26.070874 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:26.070841    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdr5q\" (UniqueName: \"kubernetes.io/projected/325e002c-0628-4442-9dfc-7f9229864371-kube-api-access-kdr5q\") pod \"postgres-69fb4594cf-5q5fj\" (UID: \"325e002c-0628-4442-9dfc-7f9229864371\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q5fj"
Jul 09 02:42:26.071035 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:26.070887    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/325e002c-0628-4442-9dfc-7f9229864371-data\") pod \"postgres-69fb4594cf-5q5fj\" (UID: \"325e002c-0628-4442-9dfc-7f9229864371\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q5fj"
Jul 09 02:42:26.117712 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:26.117675    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c2a2922-e404-4002-85af-d18fdde2b5fb" path="/var/lib/kubelet/pods/7c2a2922-e404-4002-85af-d18fdde2b5fb/volumes"
Jul 09 02:42:26.171553 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:26.171479    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdr5q\" (UniqueName: \"kubernetes.io/projected/325e002c-0628-4442-9dfc-7f9229864371-kube-api-access-kdr5q\") pod \"postgres-69fb4594cf-5q5fj\" (UID: \"325e002c-0628-4442-9dfc-7f9229864371\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q5fj"
Jul 09 02:42:26.171553 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:26.171533    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/325e002c-0628-4442-9dfc-7f9229864371-data\") pod \"postgres-69fb4594cf-5q5fj\" (UID: \"325e002c-0628-4442-9dfc-7f9229864371\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q5fj"
Jul 09 02:42:26.171863 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:26.171848    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/325e002c-0628-4442-9dfc-7f9229864371-data\") pod \"postgres-69fb4594cf-5q5fj\" (UID: \"325e002c-0628-4442-9dfc-7f9229864371\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q5fj"
Jul 09 02:42:26.179021 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:26.178997    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdr5q\" (UniqueName: \"kubernetes.io/projected/325e002c-0628-4442-9dfc-7f9229864371-kube-api-access-kdr5q\") pod \"postgres-69fb4594cf-5q5fj\" (UID: \"325e002c-0628-4442-9dfc-7f9229864371\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q5fj"
Jul 09 02:42:26.219011 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:26.218980    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q5fj"
Jul 09 02:42:26.349572 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:26.349536    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5q5fj"]
Jul 09 02:42:26.352560 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:42:26.352537    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod325e002c_0628_4442_9dfc_7f9229864371.slice/crio-c6020cd2c44536e922031eed600e3d4fbc9387d70b90b809d4e2aece7a16d6c8 WatchSource:0}: Error finding container c6020cd2c44536e922031eed600e3d4fbc9387d70b90b809d4e2aece7a16d6c8: Status 404 returned error can't find the container with id c6020cd2c44536e922031eed600e3d4fbc9387d70b90b809d4e2aece7a16d6c8
Jul 09 02:42:27.168066 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:27.167985    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q5fj" event={"ID":"325e002c-0628-4442-9dfc-7f9229864371","Type":"ContainerStarted","Data":"c6020cd2c44536e922031eed600e3d4fbc9387d70b90b809d4e2aece7a16d6c8"}
Jul 09 02:42:32.195558 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:32.195524    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q5fj" event={"ID":"325e002c-0628-4442-9dfc-7f9229864371","Type":"ContainerStarted","Data":"732c86ae987d5564c50dbf00fd33fa2d9d30b55b67032952a4536a7d658e4ab7"}
Jul 09 02:42:32.195944 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:32.195660    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q5fj"
Jul 09 02:42:32.209768 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:32.209714    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q5fj" podStartSLOduration=2.422555163 podStartE2EDuration="7.209694684s" totalImagesPullingTime="4.787139521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:42:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:42:32.208625216 +0000 UTC m=+466.602428627" watchObservedRunningTime="2026-07-09 02:42:32.209694684 +0000 UTC m=+466.603498100"
Jul 09 02:42:38.229092 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:38.229061    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5q5fj"
Jul 09 02:42:38.776156 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:38.776122    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gzqgl"]
Jul 09 02:42:38.780573 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:38.780542    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gzqgl"
Jul 09 02:42:38.788531 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:38.788480    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gzqgl"]
Jul 09 02:42:38.792779 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:38.792760    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xhkms\""
Jul 09 02:42:38.890653 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:38.890612    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg6b9\" (UniqueName: \"kubernetes.io/projected/5040df38-20f1-4fae-a51d-cf4c6f640dfc-kube-api-access-rg6b9\") pod \"authorino-5d4b67c446-gzqgl\" (UID: \"5040df38-20f1-4fae-a51d-cf4c6f640dfc\") " pod="kuadrant-system/authorino-5d4b67c446-gzqgl"
Jul 09 02:42:38.991822 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:38.991779    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg6b9\" (UniqueName: \"kubernetes.io/projected/5040df38-20f1-4fae-a51d-cf4c6f640dfc-kube-api-access-rg6b9\") pod \"authorino-5d4b67c446-gzqgl\" (UID: \"5040df38-20f1-4fae-a51d-cf4c6f640dfc\") " pod="kuadrant-system/authorino-5d4b67c446-gzqgl"
Jul 09 02:42:38.999382 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:38.999356    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg6b9\" (UniqueName: \"kubernetes.io/projected/5040df38-20f1-4fae-a51d-cf4c6f640dfc-kube-api-access-rg6b9\") pod \"authorino-5d4b67c446-gzqgl\" (UID: \"5040df38-20f1-4fae-a51d-cf4c6f640dfc\") " pod="kuadrant-system/authorino-5d4b67c446-gzqgl"
Jul 09 02:42:39.010369 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:39.010345    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gzqgl"]
Jul 09 02:42:39.010590 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:39.010579    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gzqgl"
Jul 09 02:42:39.140701 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:39.140673    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gzqgl"]
Jul 09 02:42:39.143798 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:42:39.143760    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5040df38_20f1_4fae_a51d_cf4c6f640dfc.slice/crio-e31ee533d61a33c39eff1c1f44198726dd5ae8c85e1a3cd3afcd9cb00b733389 WatchSource:0}: Error finding container e31ee533d61a33c39eff1c1f44198726dd5ae8c85e1a3cd3afcd9cb00b733389: Status 404 returned error can't find the container with id e31ee533d61a33c39eff1c1f44198726dd5ae8c85e1a3cd3afcd9cb00b733389
Jul 09 02:42:39.227270 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:39.227237    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gzqgl" event={"ID":"5040df38-20f1-4fae-a51d-cf4c6f640dfc","Type":"ContainerStarted","Data":"e31ee533d61a33c39eff1c1f44198726dd5ae8c85e1a3cd3afcd9cb00b733389"}
Jul 09 02:42:42.246560 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:42.246522    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gzqgl" event={"ID":"5040df38-20f1-4fae-a51d-cf4c6f640dfc","Type":"ContainerStarted","Data":"416a496a2f9056b083a3cc298d89b1d7fd5d9ba28b5927bc9e45c458920252fd"}
Jul 09 02:42:42.247009 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:42.246549    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-gzqgl" podUID="5040df38-20f1-4fae-a51d-cf4c6f640dfc" containerName="authorino" containerID="cri-o://416a496a2f9056b083a3cc298d89b1d7fd5d9ba28b5927bc9e45c458920252fd" gracePeriod=30
Jul 09 02:42:42.260988 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:42.260947    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-gzqgl" podStartSLOduration=1.311203702 podStartE2EDuration="4.260931084s" totalImagesPullingTime="2.949727382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:42:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:42:42.259240849 +0000 UTC m=+476.653044263" watchObservedRunningTime="2026-07-09 02:42:42.260931084 +0000 UTC m=+476.654734510"
Jul 09 02:42:42.487382 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:42.487360    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gzqgl"
Jul 09 02:42:42.529394 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:42.529367    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg6b9\" (UniqueName: \"kubernetes.io/projected/5040df38-20f1-4fae-a51d-cf4c6f640dfc-kube-api-access-rg6b9\") pod \"5040df38-20f1-4fae-a51d-cf4c6f640dfc\" (UID: \"5040df38-20f1-4fae-a51d-cf4c6f640dfc\") "
Jul 09 02:42:42.531363 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:42.531331    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5040df38-20f1-4fae-a51d-cf4c6f640dfc-kube-api-access-rg6b9" (OuterVolumeSpecName: "kube-api-access-rg6b9") pod "5040df38-20f1-4fae-a51d-cf4c6f640dfc" (UID: "5040df38-20f1-4fae-a51d-cf4c6f640dfc"). InnerVolumeSpecName "kube-api-access-rg6b9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 02:42:42.630735 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:42.630678    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg6b9\" (UniqueName: \"kubernetes.io/projected/5040df38-20f1-4fae-a51d-cf4c6f640dfc-kube-api-access-rg6b9\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:42:43.252439 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:43.252405    2578 generic.go:356] "Generic (PLEG): container finished" podID="5040df38-20f1-4fae-a51d-cf4c6f640dfc" containerID="416a496a2f9056b083a3cc298d89b1d7fd5d9ba28b5927bc9e45c458920252fd" exitCode=0
Jul 09 02:42:43.252853 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:43.252447    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gzqgl"
Jul 09 02:42:43.252853 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:43.252495    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gzqgl" event={"ID":"5040df38-20f1-4fae-a51d-cf4c6f640dfc","Type":"ContainerDied","Data":"416a496a2f9056b083a3cc298d89b1d7fd5d9ba28b5927bc9e45c458920252fd"}
Jul 09 02:42:43.252853 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:43.252546    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gzqgl" event={"ID":"5040df38-20f1-4fae-a51d-cf4c6f640dfc","Type":"ContainerDied","Data":"e31ee533d61a33c39eff1c1f44198726dd5ae8c85e1a3cd3afcd9cb00b733389"}
Jul 09 02:42:43.252853 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:43.252570    2578 scope.go:117] "RemoveContainer" containerID="416a496a2f9056b083a3cc298d89b1d7fd5d9ba28b5927bc9e45c458920252fd"
Jul 09 02:42:43.261672 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:43.261651    2578 scope.go:117] "RemoveContainer" containerID="416a496a2f9056b083a3cc298d89b1d7fd5d9ba28b5927bc9e45c458920252fd"
Jul 09 02:42:43.261961 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:42:43.261933    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"416a496a2f9056b083a3cc298d89b1d7fd5d9ba28b5927bc9e45c458920252fd\": container with ID starting with 416a496a2f9056b083a3cc298d89b1d7fd5d9ba28b5927bc9e45c458920252fd not found: ID does not exist" containerID="416a496a2f9056b083a3cc298d89b1d7fd5d9ba28b5927bc9e45c458920252fd"
Jul 09 02:42:43.262031 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:43.261960    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"416a496a2f9056b083a3cc298d89b1d7fd5d9ba28b5927bc9e45c458920252fd"} err="failed to get container status \"416a496a2f9056b083a3cc298d89b1d7fd5d9ba28b5927bc9e45c458920252fd\": rpc error: code = NotFound desc = could not find container \"416a496a2f9056b083a3cc298d89b1d7fd5d9ba28b5927bc9e45c458920252fd\": container with ID starting with 416a496a2f9056b083a3cc298d89b1d7fd5d9ba28b5927bc9e45c458920252fd not found: ID does not exist"
Jul 09 02:42:43.273085 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:43.273056    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gzqgl"]
Jul 09 02:42:43.275590 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:43.275569    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gzqgl"]
Jul 09 02:42:44.117746 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:44.117716    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5040df38-20f1-4fae-a51d-cf4c6f640dfc" path="/var/lib/kubelet/pods/5040df38-20f1-4fae-a51d-cf4c6f640dfc/volumes"
Jul 09 02:42:45.070218 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.070181    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-585b678f49-nfjs7"]
Jul 09 02:42:45.070740 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.070720    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5040df38-20f1-4fae-a51d-cf4c6f640dfc" containerName="authorino"
Jul 09 02:42:45.070831 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.070742    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5040df38-20f1-4fae-a51d-cf4c6f640dfc" containerName="authorino"
Jul 09 02:42:45.070891 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.070843    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5040df38-20f1-4fae-a51d-cf4c6f640dfc" containerName="authorino"
Jul 09 02:42:45.075413 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.075392    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-585b678f49-nfjs7"
Jul 09 02:42:45.077472 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.077452    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-v9sxg\""
Jul 09 02:42:45.077594 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.077519    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 09 02:42:45.077594 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.077535    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 09 02:42:45.083952 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.083931    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-585b678f49-nfjs7"]
Jul 09 02:42:45.152068 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.152039    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpt98\" (UniqueName: \"kubernetes.io/projected/1d864623-cc8d-420c-a3b0-fa7f8a689b79-kube-api-access-hpt98\") pod \"maas-controller-585b678f49-nfjs7\" (UID: \"1d864623-cc8d-420c-a3b0-fa7f8a689b79\") " pod="opendatahub/maas-controller-585b678f49-nfjs7"
Jul 09 02:42:45.152193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.152112    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d864623-cc8d-420c-a3b0-fa7f8a689b79-webhook-cert\") pod \"maas-controller-585b678f49-nfjs7\" (UID: \"1d864623-cc8d-420c-a3b0-fa7f8a689b79\") " pod="opendatahub/maas-controller-585b678f49-nfjs7"
Jul 09 02:42:45.253246 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.253221    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpt98\" (UniqueName: \"kubernetes.io/projected/1d864623-cc8d-420c-a3b0-fa7f8a689b79-kube-api-access-hpt98\") pod \"maas-controller-585b678f49-nfjs7\" (UID: \"1d864623-cc8d-420c-a3b0-fa7f8a689b79\") " pod="opendatahub/maas-controller-585b678f49-nfjs7"
Jul 09 02:42:45.253409 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.253280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d864623-cc8d-420c-a3b0-fa7f8a689b79-webhook-cert\") pod \"maas-controller-585b678f49-nfjs7\" (UID: \"1d864623-cc8d-420c-a3b0-fa7f8a689b79\") " pod="opendatahub/maas-controller-585b678f49-nfjs7"
Jul 09 02:42:45.254103 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.254084    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c5fc988df-f28sl"]
Jul 09 02:42:45.255894 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.255872    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d864623-cc8d-420c-a3b0-fa7f8a689b79-webhook-cert\") pod \"maas-controller-585b678f49-nfjs7\" (UID: \"1d864623-cc8d-420c-a3b0-fa7f8a689b79\") " pod="opendatahub/maas-controller-585b678f49-nfjs7"
Jul 09 02:42:45.257523 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.257508    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c5fc988df-f28sl"
Jul 09 02:42:45.263018 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.262996    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpt98\" (UniqueName: \"kubernetes.io/projected/1d864623-cc8d-420c-a3b0-fa7f8a689b79-kube-api-access-hpt98\") pod \"maas-controller-585b678f49-nfjs7\" (UID: \"1d864623-cc8d-420c-a3b0-fa7f8a689b79\") " pod="opendatahub/maas-controller-585b678f49-nfjs7"
Jul 09 02:42:45.265272 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.265252    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c5fc988df-f28sl"]
Jul 09 02:42:45.354594 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.354565    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9204e949-bc8e-4d39-96da-1dfd97a63398-webhook-cert\") pod \"maas-controller-5c5fc988df-f28sl\" (UID: \"9204e949-bc8e-4d39-96da-1dfd97a63398\") " pod="opendatahub/maas-controller-5c5fc988df-f28sl"
Jul 09 02:42:45.354703 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.354632    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq24l\" (UniqueName: \"kubernetes.io/projected/9204e949-bc8e-4d39-96da-1dfd97a63398-kube-api-access-rq24l\") pod \"maas-controller-5c5fc988df-f28sl\" (UID: \"9204e949-bc8e-4d39-96da-1dfd97a63398\") " pod="opendatahub/maas-controller-5c5fc988df-f28sl"
Jul 09 02:42:45.385589 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.385567    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-585b678f49-nfjs7"
Jul 09 02:42:45.455655 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.455618    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq24l\" (UniqueName: \"kubernetes.io/projected/9204e949-bc8e-4d39-96da-1dfd97a63398-kube-api-access-rq24l\") pod \"maas-controller-5c5fc988df-f28sl\" (UID: \"9204e949-bc8e-4d39-96da-1dfd97a63398\") " pod="opendatahub/maas-controller-5c5fc988df-f28sl"
Jul 09 02:42:45.455783 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.455691    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9204e949-bc8e-4d39-96da-1dfd97a63398-webhook-cert\") pod \"maas-controller-5c5fc988df-f28sl\" (UID: \"9204e949-bc8e-4d39-96da-1dfd97a63398\") " pod="opendatahub/maas-controller-5c5fc988df-f28sl"
Jul 09 02:42:45.457910 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.457885    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9204e949-bc8e-4d39-96da-1dfd97a63398-webhook-cert\") pod \"maas-controller-5c5fc988df-f28sl\" (UID: \"9204e949-bc8e-4d39-96da-1dfd97a63398\") " pod="opendatahub/maas-controller-5c5fc988df-f28sl"
Jul 09 02:42:45.462961 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.462938    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq24l\" (UniqueName: \"kubernetes.io/projected/9204e949-bc8e-4d39-96da-1dfd97a63398-kube-api-access-rq24l\") pod \"maas-controller-5c5fc988df-f28sl\" (UID: \"9204e949-bc8e-4d39-96da-1dfd97a63398\") " pod="opendatahub/maas-controller-5c5fc988df-f28sl"
Jul 09 02:42:45.514220 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.514194    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-585b678f49-nfjs7"]
Jul 09 02:42:45.516807 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:42:45.516783    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d864623_cc8d_420c_a3b0_fa7f8a689b79.slice/crio-e98f7ee9b67624fa853d606934cb3520db401cd7e679f312e61eff18c7f6e9c9 WatchSource:0}: Error finding container e98f7ee9b67624fa853d606934cb3520db401cd7e679f312e61eff18c7f6e9c9: Status 404 returned error can't find the container with id e98f7ee9b67624fa853d606934cb3520db401cd7e679f312e61eff18c7f6e9c9
Jul 09 02:42:45.580942 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.580918    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c5fc988df-f28sl"
Jul 09 02:42:45.705403 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:45.705370    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c5fc988df-f28sl"]
Jul 09 02:42:45.708682 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:42:45.708655    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9204e949_bc8e_4d39_96da_1dfd97a63398.slice/crio-fbe9816a04fbf264e3a198bde44113e9a3562435370a0823f38a76d0814b1860 WatchSource:0}: Error finding container fbe9816a04fbf264e3a198bde44113e9a3562435370a0823f38a76d0814b1860: Status 404 returned error can't find the container with id fbe9816a04fbf264e3a198bde44113e9a3562435370a0823f38a76d0814b1860
Jul 09 02:42:46.274889 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:46.274801    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5fc988df-f28sl" event={"ID":"9204e949-bc8e-4d39-96da-1dfd97a63398","Type":"ContainerStarted","Data":"fbe9816a04fbf264e3a198bde44113e9a3562435370a0823f38a76d0814b1860"}
Jul 09 02:42:46.277023 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:46.276979    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-585b678f49-nfjs7" event={"ID":"1d864623-cc8d-420c-a3b0-fa7f8a689b79","Type":"ContainerStarted","Data":"e98f7ee9b67624fa853d606934cb3520db401cd7e679f312e61eff18c7f6e9c9"}
Jul 09 02:42:47.678028 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:47.678002    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 09 02:42:48.289307 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:48.289264    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5fc988df-f28sl" event={"ID":"9204e949-bc8e-4d39-96da-1dfd97a63398","Type":"ContainerStarted","Data":"8d377fce6dc58b1547e8050a25a8ff945d19162e514440db1448345e242239f4"}
Jul 09 02:42:48.289538 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:48.289346    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c5fc988df-f28sl"
Jul 09 02:42:48.290805 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:48.290781    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-585b678f49-nfjs7" event={"ID":"1d864623-cc8d-420c-a3b0-fa7f8a689b79","Type":"ContainerStarted","Data":"97a6d9e695333fda8ce94022e551f24caf59a24a151215fe77d5b1084a65a4dc"}
Jul 09 02:42:48.290920 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:48.290909    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-585b678f49-nfjs7"
Jul 09 02:42:48.309838 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:48.309792    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c5fc988df-f28sl" podStartSLOduration=1.3439512439999999 podStartE2EDuration="3.309777408s" totalImagesPullingTime="1.965826164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:42:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:42:48.30845138 +0000 UTC m=+482.702254793" watchObservedRunningTime="2026-07-09 02:42:48.309777408 +0000 UTC m=+482.703580821"
Jul 09 02:42:48.326998 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:48.326955    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-585b678f49-nfjs7" podStartSLOduration=1.162199194 podStartE2EDuration="3.326940788s" totalImagesPullingTime="2.164741594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:42:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:42:48.325375403 +0000 UTC m=+482.719178818" watchObservedRunningTime="2026-07-09 02:42:48.326940788 +0000 UTC m=+482.720744201"
Jul 09 02:42:59.302033 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:59.301999    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-585b678f49-nfjs7"
Jul 09 02:42:59.302420 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:59.302080    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c5fc988df-f28sl"
Jul 09 02:42:59.355003 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:59.354970    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-585b678f49-nfjs7"]
Jul 09 02:42:59.355279 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:59.355207    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-585b678f49-nfjs7" podUID="1d864623-cc8d-420c-a3b0-fa7f8a689b79" containerName="manager" containerID="cri-o://97a6d9e695333fda8ce94022e551f24caf59a24a151215fe77d5b1084a65a4dc" gracePeriod=30
Jul 09 02:42:59.597470 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:59.597447    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-585b678f49-nfjs7"
Jul 09 02:42:59.680799 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:59.680764    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpt98\" (UniqueName: \"kubernetes.io/projected/1d864623-cc8d-420c-a3b0-fa7f8a689b79-kube-api-access-hpt98\") pod \"1d864623-cc8d-420c-a3b0-fa7f8a689b79\" (UID: \"1d864623-cc8d-420c-a3b0-fa7f8a689b79\") "
Jul 09 02:42:59.680969 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:59.680870    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d864623-cc8d-420c-a3b0-fa7f8a689b79-webhook-cert\") pod \"1d864623-cc8d-420c-a3b0-fa7f8a689b79\" (UID: \"1d864623-cc8d-420c-a3b0-fa7f8a689b79\") "
Jul 09 02:42:59.682953 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:59.682918    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d864623-cc8d-420c-a3b0-fa7f8a689b79-kube-api-access-hpt98" (OuterVolumeSpecName: "kube-api-access-hpt98") pod "1d864623-cc8d-420c-a3b0-fa7f8a689b79" (UID: "1d864623-cc8d-420c-a3b0-fa7f8a689b79"). InnerVolumeSpecName "kube-api-access-hpt98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 02:42:59.683069 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:59.682951    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d864623-cc8d-420c-a3b0-fa7f8a689b79-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1d864623-cc8d-420c-a3b0-fa7f8a689b79" (UID: "1d864623-cc8d-420c-a3b0-fa7f8a689b79"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 02:42:59.782399 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:59.782363    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d864623-cc8d-420c-a3b0-fa7f8a689b79-webhook-cert\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:42:59.782399 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:42:59.782394    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpt98\" (UniqueName: \"kubernetes.io/projected/1d864623-cc8d-420c-a3b0-fa7f8a689b79-kube-api-access-hpt98\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:43:00.347171 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:00.347135    2578 generic.go:356] "Generic (PLEG): container finished" podID="1d864623-cc8d-420c-a3b0-fa7f8a689b79" containerID="97a6d9e695333fda8ce94022e551f24caf59a24a151215fe77d5b1084a65a4dc" exitCode=0
Jul 09 02:43:00.347648 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:00.347180    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-585b678f49-nfjs7"
Jul 09 02:43:00.347648 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:00.347215    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-585b678f49-nfjs7" event={"ID":"1d864623-cc8d-420c-a3b0-fa7f8a689b79","Type":"ContainerDied","Data":"97a6d9e695333fda8ce94022e551f24caf59a24a151215fe77d5b1084a65a4dc"}
Jul 09 02:43:00.347648 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:00.347281    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-585b678f49-nfjs7" event={"ID":"1d864623-cc8d-420c-a3b0-fa7f8a689b79","Type":"ContainerDied","Data":"e98f7ee9b67624fa853d606934cb3520db401cd7e679f312e61eff18c7f6e9c9"}
Jul 09 02:43:00.347648 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:00.347336    2578 scope.go:117] "RemoveContainer" containerID="97a6d9e695333fda8ce94022e551f24caf59a24a151215fe77d5b1084a65a4dc"
Jul 09 02:43:00.355640 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:00.355618    2578 scope.go:117] "RemoveContainer" containerID="97a6d9e695333fda8ce94022e551f24caf59a24a151215fe77d5b1084a65a4dc"
Jul 09 02:43:00.355882 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:43:00.355859    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97a6d9e695333fda8ce94022e551f24caf59a24a151215fe77d5b1084a65a4dc\": container with ID starting with 97a6d9e695333fda8ce94022e551f24caf59a24a151215fe77d5b1084a65a4dc not found: ID does not exist" containerID="97a6d9e695333fda8ce94022e551f24caf59a24a151215fe77d5b1084a65a4dc"
Jul 09 02:43:00.355961 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:00.355888    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a6d9e695333fda8ce94022e551f24caf59a24a151215fe77d5b1084a65a4dc"} err="failed to get container status \"97a6d9e695333fda8ce94022e551f24caf59a24a151215fe77d5b1084a65a4dc\": rpc error: code = NotFound desc = could not find container \"97a6d9e695333fda8ce94022e551f24caf59a24a151215fe77d5b1084a65a4dc\": container with ID starting with 97a6d9e695333fda8ce94022e551f24caf59a24a151215fe77d5b1084a65a4dc not found: ID does not exist"
Jul 09 02:43:00.361998 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:00.361976    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-585b678f49-nfjs7"]
Jul 09 02:43:00.364206 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:00.364186    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-585b678f49-nfjs7"]
Jul 09 02:43:02.118632 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:02.118594    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d864623-cc8d-420c-a3b0-fa7f8a689b79" path="/var/lib/kubelet/pods/1d864623-cc8d-420c-a3b0-fa7f8a689b79/volumes"
Jul 09 02:43:20.899178 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:20.899142    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"]
Jul 09 02:43:20.899656 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:20.899528    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d864623-cc8d-420c-a3b0-fa7f8a689b79" containerName="manager"
Jul 09 02:43:20.899656 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:20.899540    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d864623-cc8d-420c-a3b0-fa7f8a689b79" containerName="manager"
Jul 09 02:43:20.899656 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:20.899609    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d864623-cc8d-420c-a3b0-fa7f8a689b79" containerName="manager"
Jul 09 02:43:20.903541 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:20.903520    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"
Jul 09 02:43:20.905488 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:20.905457    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 09 02:43:20.905622 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:20.905581    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p7bw6\""
Jul 09 02:43:20.913155 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:20.913130    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"]
Jul 09 02:43:21.071692 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.071660    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e668c4c1-fd2c-47a8-ad1f-34ddef904cb8-maas-api-tls\") pod \"maas-api-6b5665c595-zg7fm\" (UID: \"e668c4c1-fd2c-47a8-ad1f-34ddef904cb8\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"
Jul 09 02:43:21.071862 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.071713    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e668c4c1-fd2c-47a8-ad1f-34ddef904cb8-openshift-service-ca\") pod \"maas-api-6b5665c595-zg7fm\" (UID: \"e668c4c1-fd2c-47a8-ad1f-34ddef904cb8\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"
Jul 09 02:43:21.071862 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.071837    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlcnv\" (UniqueName: \"kubernetes.io/projected/e668c4c1-fd2c-47a8-ad1f-34ddef904cb8-kube-api-access-dlcnv\") pod \"maas-api-6b5665c595-zg7fm\" (UID: \"e668c4c1-fd2c-47a8-ad1f-34ddef904cb8\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"
Jul 09 02:43:21.172811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.172725    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e668c4c1-fd2c-47a8-ad1f-34ddef904cb8-openshift-service-ca\") pod \"maas-api-6b5665c595-zg7fm\" (UID: \"e668c4c1-fd2c-47a8-ad1f-34ddef904cb8\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"
Jul 09 02:43:21.172811 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.172796    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlcnv\" (UniqueName: \"kubernetes.io/projected/e668c4c1-fd2c-47a8-ad1f-34ddef904cb8-kube-api-access-dlcnv\") pod \"maas-api-6b5665c595-zg7fm\" (UID: \"e668c4c1-fd2c-47a8-ad1f-34ddef904cb8\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"
Jul 09 02:43:21.172991 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.172840    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e668c4c1-fd2c-47a8-ad1f-34ddef904cb8-maas-api-tls\") pod \"maas-api-6b5665c595-zg7fm\" (UID: \"e668c4c1-fd2c-47a8-ad1f-34ddef904cb8\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"
Jul 09 02:43:21.172991 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:43:21.172945    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 09 02:43:21.173094 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:43:21.173056    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e668c4c1-fd2c-47a8-ad1f-34ddef904cb8-maas-api-tls podName:e668c4c1-fd2c-47a8-ad1f-34ddef904cb8 nodeName:}" failed. No retries permitted until 2026-07-09 02:43:21.673034917 +0000 UTC m=+516.066838310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e668c4c1-fd2c-47a8-ad1f-34ddef904cb8-maas-api-tls") pod "maas-api-6b5665c595-zg7fm" (UID: "e668c4c1-fd2c-47a8-ad1f-34ddef904cb8") : secret "maas-api-serving-cert" not found
Jul 09 02:43:21.173413 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.173391    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e668c4c1-fd2c-47a8-ad1f-34ddef904cb8-openshift-service-ca\") pod \"maas-api-6b5665c595-zg7fm\" (UID: \"e668c4c1-fd2c-47a8-ad1f-34ddef904cb8\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"
Jul 09 02:43:21.180595 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.180570    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlcnv\" (UniqueName: \"kubernetes.io/projected/e668c4c1-fd2c-47a8-ad1f-34ddef904cb8-kube-api-access-dlcnv\") pod \"maas-api-6b5665c595-zg7fm\" (UID: \"e668c4c1-fd2c-47a8-ad1f-34ddef904cb8\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"
Jul 09 02:43:21.449267 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.449194    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-k5674"]
Jul 09 02:43:21.455847 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.455826    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-k5674"
Jul 09 02:43:21.458024 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.457998    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 09 02:43:21.458141 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.458120    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-vtn9l\""
Jul 09 02:43:21.463645 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.463625    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-k5674"]
Jul 09 02:43:21.499818 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.499793    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-s4mtn"]
Jul 09 02:43:21.506541 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.506525    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-s4mtn"
Jul 09 02:43:21.517854 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.517828    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-s4mtn"]
Jul 09 02:43:21.576916 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.576892    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d59f60c5-7096-4144-bb66-41770698c788-plugins-config-volume\") pod \"payload-pre-processing-7964457696-k5674\" (UID: \"d59f60c5-7096-4144-bb66-41770698c788\") " pod="openshift-ingress/payload-pre-processing-7964457696-k5674"
Jul 09 02:43:21.576916 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.576925    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8xv5\" (UniqueName: \"kubernetes.io/projected/d59f60c5-7096-4144-bb66-41770698c788-kube-api-access-d8xv5\") pod \"payload-pre-processing-7964457696-k5674\" (UID: \"d59f60c5-7096-4144-bb66-41770698c788\") " pod="openshift-ingress/payload-pre-processing-7964457696-k5674"
Jul 09 02:43:21.677892 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.677858    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e668c4c1-fd2c-47a8-ad1f-34ddef904cb8-maas-api-tls\") pod \"maas-api-6b5665c595-zg7fm\" (UID: \"e668c4c1-fd2c-47a8-ad1f-34ddef904cb8\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"
Jul 09 02:43:21.678038 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.677897    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4dc21f4-39f2-4826-927e-eda9227e102f-plugins-config-volume\") pod \"payload-processing-58f86c7798-s4mtn\" (UID: \"a4dc21f4-39f2-4826-927e-eda9227e102f\") " pod="openshift-ingress/payload-processing-58f86c7798-s4mtn"
Jul 09 02:43:21.678038 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.677990    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d59f60c5-7096-4144-bb66-41770698c788-plugins-config-volume\") pod \"payload-pre-processing-7964457696-k5674\" (UID: \"d59f60c5-7096-4144-bb66-41770698c788\") " pod="openshift-ingress/payload-pre-processing-7964457696-k5674"
Jul 09 02:43:21.678138 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.678061    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8xv5\" (UniqueName: \"kubernetes.io/projected/d59f60c5-7096-4144-bb66-41770698c788-kube-api-access-d8xv5\") pod \"payload-pre-processing-7964457696-k5674\" (UID: \"d59f60c5-7096-4144-bb66-41770698c788\") " pod="openshift-ingress/payload-pre-processing-7964457696-k5674"
Jul 09 02:43:21.678138 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.678096    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrpjq\" (UniqueName: \"kubernetes.io/projected/a4dc21f4-39f2-4826-927e-eda9227e102f-kube-api-access-hrpjq\") pod \"payload-processing-58f86c7798-s4mtn\" (UID: \"a4dc21f4-39f2-4826-927e-eda9227e102f\") " pod="openshift-ingress/payload-processing-58f86c7798-s4mtn"
Jul 09 02:43:21.678681 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.678655    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d59f60c5-7096-4144-bb66-41770698c788-plugins-config-volume\") pod \"payload-pre-processing-7964457696-k5674\" (UID: \"d59f60c5-7096-4144-bb66-41770698c788\") " pod="openshift-ingress/payload-pre-processing-7964457696-k5674"
Jul 09 02:43:21.680419 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.680392    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e668c4c1-fd2c-47a8-ad1f-34ddef904cb8-maas-api-tls\") pod \"maas-api-6b5665c595-zg7fm\" (UID: \"e668c4c1-fd2c-47a8-ad1f-34ddef904cb8\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"
Jul 09 02:43:21.685256 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.685236    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8xv5\" (UniqueName: \"kubernetes.io/projected/d59f60c5-7096-4144-bb66-41770698c788-kube-api-access-d8xv5\") pod \"payload-pre-processing-7964457696-k5674\" (UID: \"d59f60c5-7096-4144-bb66-41770698c788\") " pod="openshift-ingress/payload-pre-processing-7964457696-k5674"
Jul 09 02:43:21.768810 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.768746    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-k5674"
Jul 09 02:43:21.779369 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.779339    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrpjq\" (UniqueName: \"kubernetes.io/projected/a4dc21f4-39f2-4826-927e-eda9227e102f-kube-api-access-hrpjq\") pod \"payload-processing-58f86c7798-s4mtn\" (UID: \"a4dc21f4-39f2-4826-927e-eda9227e102f\") " pod="openshift-ingress/payload-processing-58f86c7798-s4mtn"
Jul 09 02:43:21.779482 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.779400    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4dc21f4-39f2-4826-927e-eda9227e102f-plugins-config-volume\") pod \"payload-processing-58f86c7798-s4mtn\" (UID: \"a4dc21f4-39f2-4826-927e-eda9227e102f\") " pod="openshift-ingress/payload-processing-58f86c7798-s4mtn"
Jul 09 02:43:21.780220 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.780189    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4dc21f4-39f2-4826-927e-eda9227e102f-plugins-config-volume\") pod \"payload-processing-58f86c7798-s4mtn\" (UID: \"a4dc21f4-39f2-4826-927e-eda9227e102f\") " pod="openshift-ingress/payload-processing-58f86c7798-s4mtn"
Jul 09 02:43:21.786923 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.786875    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrpjq\" (UniqueName: \"kubernetes.io/projected/a4dc21f4-39f2-4826-927e-eda9227e102f-kube-api-access-hrpjq\") pod \"payload-processing-58f86c7798-s4mtn\" (UID: \"a4dc21f4-39f2-4826-927e-eda9227e102f\") " pod="openshift-ingress/payload-processing-58f86c7798-s4mtn"
Jul 09 02:43:21.814899 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.814869    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"
Jul 09 02:43:21.817895 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.817871    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-s4mtn"
Jul 09 02:43:21.932876 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.932841    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-k5674"]
Jul 09 02:43:21.934613 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:43:21.934585    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd59f60c5_7096_4144_bb66_41770698c788.slice/crio-f2542ee84868b686285fc8bc2217e33337191a4bc64689d9503a137bcc3a4771 WatchSource:0}: Error finding container f2542ee84868b686285fc8bc2217e33337191a4bc64689d9503a137bcc3a4771: Status 404 returned error can't find the container with id f2542ee84868b686285fc8bc2217e33337191a4bc64689d9503a137bcc3a4771
Jul 09 02:43:21.973761 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.973731    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"]
Jul 09 02:43:21.978409 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:43:21.978377    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode668c4c1_fd2c_47a8_ad1f_34ddef904cb8.slice/crio-b74ca3921a272f3383758aba505be1df2cf175aafc3eebbe6aafc35decf3efd2 WatchSource:0}: Error finding container b74ca3921a272f3383758aba505be1df2cf175aafc3eebbe6aafc35decf3efd2: Status 404 returned error can't find the container with id b74ca3921a272f3383758aba505be1df2cf175aafc3eebbe6aafc35decf3efd2
Jul 09 02:43:21.995828 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:21.995775    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-s4mtn"]
Jul 09 02:43:21.998089 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:43:21.998067    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4dc21f4_39f2_4826_927e_eda9227e102f.slice/crio-c2a42f1010ddf32e1216f98db9c6785766fe7d7e92beb0a77bf90319e1936a6d WatchSource:0}: Error finding container c2a42f1010ddf32e1216f98db9c6785766fe7d7e92beb0a77bf90319e1936a6d: Status 404 returned error can't find the container with id c2a42f1010ddf32e1216f98db9c6785766fe7d7e92beb0a77bf90319e1936a6d
Jul 09 02:43:22.455013 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:22.454980    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm" event={"ID":"e668c4c1-fd2c-47a8-ad1f-34ddef904cb8","Type":"ContainerStarted","Data":"b74ca3921a272f3383758aba505be1df2cf175aafc3eebbe6aafc35decf3efd2"}
Jul 09 02:43:22.456093 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:22.456068    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-k5674" event={"ID":"d59f60c5-7096-4144-bb66-41770698c788","Type":"ContainerStarted","Data":"f2542ee84868b686285fc8bc2217e33337191a4bc64689d9503a137bcc3a4771"}
Jul 09 02:43:22.457098 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:22.457079    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-s4mtn" event={"ID":"a4dc21f4-39f2-4826-927e-eda9227e102f","Type":"ContainerStarted","Data":"c2a42f1010ddf32e1216f98db9c6785766fe7d7e92beb0a77bf90319e1936a6d"}
Jul 09 02:43:23.465551 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:23.465511    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-k5674" event={"ID":"d59f60c5-7096-4144-bb66-41770698c788","Type":"ContainerStarted","Data":"ccd51aa53b75a4534d230aa76dc684c61517216e16f183348a87565ef8667c10"}
Jul 09 02:43:23.465998 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:23.465782    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-k5674"
Jul 09 02:43:23.467503 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:23.467448    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-s4mtn" event={"ID":"a4dc21f4-39f2-4826-927e-eda9227e102f","Type":"ContainerStarted","Data":"8545331cbad3a59d5a1544072f7b950be132ebd633266f8f2fa5a4aca44096bc"}
Jul 09 02:43:23.467619 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:23.467554    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-s4mtn"
Jul 09 02:43:23.483058 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:23.483001    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-k5674" podStartSLOduration=1.072239451 podStartE2EDuration="2.482978964s" totalImagesPullingTime="1.410739513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:43:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:43:23.480043569 +0000 UTC m=+517.873846984" watchObservedRunningTime="2026-07-09 02:43:23.482978964 +0000 UTC m=+517.876782378"
Jul 09 02:43:23.498686 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:23.498632    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-s4mtn" podStartSLOduration=1.137940709 podStartE2EDuration="2.498618545s" totalImagesPullingTime="1.360677836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:43:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:43:23.495860009 +0000 UTC m=+517.889663423" watchObservedRunningTime="2026-07-09 02:43:23.498618545 +0000 UTC m=+517.892421935"
Jul 09 02:43:24.474410 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:24.474372    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm" event={"ID":"e668c4c1-fd2c-47a8-ad1f-34ddef904cb8","Type":"ContainerStarted","Data":"fbcc70fa5d21db4c817b9ec8d4741ea6df9a09cb3abc3462b1a6460cf7855d12"}
Jul 09 02:43:24.474837 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:24.474777    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"
Jul 09 02:43:24.490036 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:24.489950    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm" podStartSLOduration=2.557770553 podStartE2EDuration="4.489934482s" totalImagesPullingTime="1.932163929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:43:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:43:24.489083544 +0000 UTC m=+518.882886947" watchObservedRunningTime="2026-07-09 02:43:24.489934482 +0000 UTC m=+518.883737896"
Jul 09 02:43:31.491186 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:31.491157    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6b5665c595-zg7fm"
Jul 09 02:43:34.476528 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:34.476495    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-s4mtn"
Jul 09 02:43:34.477002 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:43:34.476829    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-k5674"
Jul 09 02:44:13.377219 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.377182    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"]
Jul 09 02:44:13.381390 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.381368    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.384127 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.384102    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 09 02:44:13.384261 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.384101    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 09 02:44:13.384261 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.384103    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vnpf7\""
Jul 09 02:44:13.384261 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.384102    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 09 02:44:13.391620 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.391598    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"]
Jul 09 02:44:13.524807 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.524772    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dccd6202-276b-445d-8853-900bd0daa99c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.524807 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.524811    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dccd6202-276b-445d-8853-900bd0daa99c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.525017 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.524847    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dccd6202-276b-445d-8853-900bd0daa99c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.525017 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.524897    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dccd6202-276b-445d-8853-900bd0daa99c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.525017 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.524914    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dccd6202-276b-445d-8853-900bd0daa99c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.525017 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.524937    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq8b5\" (UniqueName: \"kubernetes.io/projected/dccd6202-276b-445d-8853-900bd0daa99c-kube-api-access-mq8b5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.625638 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.625609    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dccd6202-276b-445d-8853-900bd0daa99c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.625808 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.625653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dccd6202-276b-445d-8853-900bd0daa99c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.625808 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.625670    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dccd6202-276b-445d-8853-900bd0daa99c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.625808 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.625690    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq8b5\" (UniqueName: \"kubernetes.io/projected/dccd6202-276b-445d-8853-900bd0daa99c-kube-api-access-mq8b5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.625808 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.625758    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dccd6202-276b-445d-8853-900bd0daa99c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.625808 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.625788    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dccd6202-276b-445d-8853-900bd0daa99c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.626151 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.626123    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dccd6202-276b-445d-8853-900bd0daa99c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.626233 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.626206    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dccd6202-276b-445d-8853-900bd0daa99c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.626301 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.626271    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dccd6202-276b-445d-8853-900bd0daa99c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.628040 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.627994    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dccd6202-276b-445d-8853-900bd0daa99c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.628245 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.628221    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dccd6202-276b-445d-8853-900bd0daa99c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.633149 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.633128    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq8b5\" (UniqueName: \"kubernetes.io/projected/dccd6202-276b-445d-8853-900bd0daa99c-kube-api-access-mq8b5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5xr2w\" (UID: \"dccd6202-276b-445d-8853-900bd0daa99c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.693171 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.693145    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:13.835120 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:13.835095    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"]
Jul 09 02:44:13.837800 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:44:13.837774    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddccd6202_276b_445d_8853_900bd0daa99c.slice/crio-8d62d302aae29e2b231fc3610bee6277e6456e75253badd4ceb2fba420bba5e3 WatchSource:0}: Error finding container 8d62d302aae29e2b231fc3610bee6277e6456e75253badd4ceb2fba420bba5e3: Status 404 returned error can't find the container with id 8d62d302aae29e2b231fc3610bee6277e6456e75253badd4ceb2fba420bba5e3
Jul 09 02:44:14.715691 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:14.715643    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w" event={"ID":"dccd6202-276b-445d-8853-900bd0daa99c","Type":"ContainerStarted","Data":"8d62d302aae29e2b231fc3610bee6277e6456e75253badd4ceb2fba420bba5e3"}
Jul 09 02:44:15.722587 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:15.722552    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w" event={"ID":"dccd6202-276b-445d-8853-900bd0daa99c","Type":"ContainerStarted","Data":"f0ffeccd7443b4f586648efc7032d5ef56ba978a02fd563ef6d67b720f06cfcb"}
Jul 09 02:44:15.738366 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:15.738297    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w" podStartSLOduration=1.641807241 podStartE2EDuration="2.738279849s" totalImagesPullingTime="1.096472608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:44:15.737497029 +0000 UTC m=+570.131300443" watchObservedRunningTime="2026-07-09 02:44:15.738279849 +0000 UTC m=+570.132083264"
Jul 09 02:44:23.693990 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:23.693953    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:23.694500 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:23.694020    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:23.706459 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:23.706432    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:23.772458 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:23.772431    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5xr2w"
Jul 09 02:44:36.287558 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.287524    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"]
Jul 09 02:44:36.294487 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.294467    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.296556 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.296536    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 09 02:44:36.301309 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.301289    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"]
Jul 09 02:44:36.421450 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.421414    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.421629 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.421482    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.421629 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.421534    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s28rx\" (UniqueName: \"kubernetes.io/projected/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-kube-api-access-s28rx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.421629 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.421589    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.421629 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.421611    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.421771 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.421675    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.522381 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.522345    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.522381 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.522383    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.522621 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.522415    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.522621 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.522445    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.522621 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.522593    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.522784 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.522652    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s28rx\" (UniqueName: \"kubernetes.io/projected/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-kube-api-access-s28rx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.522839 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.522781    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.522839 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.522807    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.522928 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.522911    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.525236 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.525210    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.525471 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.525450    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.530368 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.530343    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s28rx\" (UniqueName: \"kubernetes.io/projected/50b17a53-46ea-4987-b4ae-7cd74cfa05f0-kube-api-access-s28rx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm\" (UID: \"50b17a53-46ea-4987-b4ae-7cd74cfa05f0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.607183 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.607146    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:36.766865 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.766840    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"]
Jul 09 02:44:36.769183 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:44:36.769152    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50b17a53_46ea_4987_b4ae_7cd74cfa05f0.slice/crio-ed94bf4a1703d5f5764b63614b9249d74f8c72208fae2c0f0951612ec97c1a2e WatchSource:0}: Error finding container ed94bf4a1703d5f5764b63614b9249d74f8c72208fae2c0f0951612ec97c1a2e: Status 404 returned error can't find the container with id ed94bf4a1703d5f5764b63614b9249d74f8c72208fae2c0f0951612ec97c1a2e
Jul 09 02:44:36.823770 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:36.823740    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm" event={"ID":"50b17a53-46ea-4987-b4ae-7cd74cfa05f0","Type":"ContainerStarted","Data":"ed94bf4a1703d5f5764b63614b9249d74f8c72208fae2c0f0951612ec97c1a2e"}
Jul 09 02:44:37.829795 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:37.829755    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm" event={"ID":"50b17a53-46ea-4987-b4ae-7cd74cfa05f0","Type":"ContainerStarted","Data":"41eee9b3f3381a9d490b6ba21f70c8f3a02486a520feb2729bad8294f6c6a995"}
Jul 09 02:44:37.852532 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:37.852479    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm" podStartSLOduration=1.667618177 podStartE2EDuration="1.852463826s" totalImagesPullingTime="184.845649ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:44:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:44:37.85023304 +0000 UTC m=+592.244036455" watchObservedRunningTime="2026-07-09 02:44:37.852463826 +0000 UTC m=+592.246267238"
Jul 09 02:44:46.607813 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:46.607782    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:46.608340 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:46.607837    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:46.620009 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:46.619987    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:44:46.887684 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:44:46.887611    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm"
Jul 09 02:45:00.140847 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:45:00.140811    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p"]
Jul 09 02:45:00.144409 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:45:00.144385    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p"
Jul 09 02:45:00.150387 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:45:00.150359    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p"]
Jul 09 02:45:00.242108 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:45:00.242076    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmxx8\" (UniqueName: \"kubernetes.io/projected/ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7-kube-api-access-kmxx8\") pod \"maas-api-key-cleanup-29726085-hxb8p\" (UID: \"ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p"
Jul 09 02:45:00.342981 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:45:00.342945    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmxx8\" (UniqueName: \"kubernetes.io/projected/ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7-kube-api-access-kmxx8\") pod \"maas-api-key-cleanup-29726085-hxb8p\" (UID: \"ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p"
Jul 09 02:45:00.350836 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:45:00.350811    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmxx8\" (UniqueName: \"kubernetes.io/projected/ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7-kube-api-access-kmxx8\") pod \"maas-api-key-cleanup-29726085-hxb8p\" (UID: \"ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p"
Jul 09 02:45:00.456959 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:45:00.456869    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p"
Jul 09 02:45:00.587685 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:45:00.587653    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p"]
Jul 09 02:45:00.588834 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:45:00.588804    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba27bd7f_cf5f_4a2f_a51e_00ab7cb35cf7.slice/crio-427de5c5dac002083d1b48904b6a9b32b6ca83a74c0f2e0b9d555b95e9c0f90b WatchSource:0}: Error finding container 427de5c5dac002083d1b48904b6a9b32b6ca83a74c0f2e0b9d555b95e9c0f90b: Status 404 returned error can't find the container with id 427de5c5dac002083d1b48904b6a9b32b6ca83a74c0f2e0b9d555b95e9c0f90b
Jul 09 02:45:00.952141 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:45:00.952108    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p" event={"ID":"ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7","Type":"ContainerStarted","Data":"427de5c5dac002083d1b48904b6a9b32b6ca83a74c0f2e0b9d555b95e9c0f90b"}
Jul 09 02:45:03.969191 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:45:03.969153    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p" event={"ID":"ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7","Type":"ContainerStarted","Data":"a20e20bae3e97a7416f7a70469b00737e18e097a79d41e36b28728a2cdca4d83"}
Jul 09 02:45:03.983594 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:45:03.983547    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p" podStartSLOduration=1.5249499420000001 podStartE2EDuration="3.983531757s" totalImagesPullingTime="2.458581815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:45:03.982748532 +0000 UTC m=+618.376551945" watchObservedRunningTime="2026-07-09 02:45:03.983531757 +0000 UTC m=+618.377335215"
Jul 09 02:46:00.013394 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:00.013353    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p"]
Jul 09 02:46:00.013944 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:00.013571    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p" podUID="ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7" containerName="cleanup" containerID="cri-o://a20e20bae3e97a7416f7a70469b00737e18e097a79d41e36b28728a2cdca4d83" gracePeriod=30
Jul 09 02:46:30.192217 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:30.192189    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p"
Jul 09 02:46:30.363895 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:30.363866    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmxx8\" (UniqueName: \"kubernetes.io/projected/ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7-kube-api-access-kmxx8\") pod \"ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7\" (UID: \"ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7\") "
Jul 09 02:46:30.366069 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:30.366040    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7-kube-api-access-kmxx8" (OuterVolumeSpecName: "kube-api-access-kmxx8") pod "ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7" (UID: "ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7"). InnerVolumeSpecName "kube-api-access-kmxx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 02:46:30.387392 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:30.387355    2578 generic.go:356] "Generic (PLEG): container finished" podID="ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7" containerID="a20e20bae3e97a7416f7a70469b00737e18e097a79d41e36b28728a2cdca4d83" exitCode=137
Jul 09 02:46:30.387521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:30.387399    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p"
Jul 09 02:46:30.387521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:30.387444    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p" event={"ID":"ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7","Type":"ContainerDied","Data":"a20e20bae3e97a7416f7a70469b00737e18e097a79d41e36b28728a2cdca4d83"}
Jul 09 02:46:30.387521 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:30.387509    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p" event={"ID":"ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7","Type":"ContainerDied","Data":"427de5c5dac002083d1b48904b6a9b32b6ca83a74c0f2e0b9d555b95e9c0f90b"}
Jul 09 02:46:30.387683 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:30.387541    2578 scope.go:117] "RemoveContainer" containerID="a20e20bae3e97a7416f7a70469b00737e18e097a79d41e36b28728a2cdca4d83"
Jul 09 02:46:30.396507 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:30.396490    2578 scope.go:117] "RemoveContainer" containerID="a20e20bae3e97a7416f7a70469b00737e18e097a79d41e36b28728a2cdca4d83"
Jul 09 02:46:30.396801 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:46:30.396775    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a20e20bae3e97a7416f7a70469b00737e18e097a79d41e36b28728a2cdca4d83\": container with ID starting with a20e20bae3e97a7416f7a70469b00737e18e097a79d41e36b28728a2cdca4d83 not found: ID does not exist" containerID="a20e20bae3e97a7416f7a70469b00737e18e097a79d41e36b28728a2cdca4d83"
Jul 09 02:46:30.396849 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:30.396808    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a20e20bae3e97a7416f7a70469b00737e18e097a79d41e36b28728a2cdca4d83"} err="failed to get container status \"a20e20bae3e97a7416f7a70469b00737e18e097a79d41e36b28728a2cdca4d83\": rpc error: code = NotFound desc = could not find container \"a20e20bae3e97a7416f7a70469b00737e18e097a79d41e36b28728a2cdca4d83\": container with ID starting with a20e20bae3e97a7416f7a70469b00737e18e097a79d41e36b28728a2cdca4d83 not found: ID does not exist"
Jul 09 02:46:30.407218 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:30.407193    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p"]
Jul 09 02:46:30.410763 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:30.410740    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726085-hxb8p"]
Jul 09 02:46:30.465514 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:30.465488    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmxx8\" (UniqueName: \"kubernetes.io/projected/ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7-kube-api-access-kmxx8\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:46:32.117897 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:32.117859    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7" path="/var/lib/kubelet/pods/ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7/volumes"
Jul 09 02:46:48.441559 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:48.441518    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c5fc988df-f28sl"]
Jul 09 02:46:48.442019 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:48.441830    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5c5fc988df-f28sl" podUID="9204e949-bc8e-4d39-96da-1dfd97a63398" containerName="manager" containerID="cri-o://8d377fce6dc58b1547e8050a25a8ff945d19162e514440db1448345e242239f4" gracePeriod=30
Jul 09 02:46:48.695508 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:48.695442    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c5fc988df-f28sl"
Jul 09 02:46:48.833442 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:48.833414    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9204e949-bc8e-4d39-96da-1dfd97a63398-webhook-cert\") pod \"9204e949-bc8e-4d39-96da-1dfd97a63398\" (UID: \"9204e949-bc8e-4d39-96da-1dfd97a63398\") "
Jul 09 02:46:48.833442 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:48.833447    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rq24l\" (UniqueName: \"kubernetes.io/projected/9204e949-bc8e-4d39-96da-1dfd97a63398-kube-api-access-rq24l\") pod \"9204e949-bc8e-4d39-96da-1dfd97a63398\" (UID: \"9204e949-bc8e-4d39-96da-1dfd97a63398\") "
Jul 09 02:46:48.835568 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:48.835529    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9204e949-bc8e-4d39-96da-1dfd97a63398-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9204e949-bc8e-4d39-96da-1dfd97a63398" (UID: "9204e949-bc8e-4d39-96da-1dfd97a63398"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 02:46:48.835709 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:48.835648    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9204e949-bc8e-4d39-96da-1dfd97a63398-kube-api-access-rq24l" (OuterVolumeSpecName: "kube-api-access-rq24l") pod "9204e949-bc8e-4d39-96da-1dfd97a63398" (UID: "9204e949-bc8e-4d39-96da-1dfd97a63398"). InnerVolumeSpecName "kube-api-access-rq24l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 02:46:48.934395 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:48.934364    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9204e949-bc8e-4d39-96da-1dfd97a63398-webhook-cert\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:46:48.934395 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:48.934394    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rq24l\" (UniqueName: \"kubernetes.io/projected/9204e949-bc8e-4d39-96da-1dfd97a63398-kube-api-access-rq24l\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:46:49.482561 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:49.482526    2578 generic.go:356] "Generic (PLEG): container finished" podID="9204e949-bc8e-4d39-96da-1dfd97a63398" containerID="8d377fce6dc58b1547e8050a25a8ff945d19162e514440db1448345e242239f4" exitCode=0
Jul 09 02:46:49.482996 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:49.482573    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c5fc988df-f28sl"
Jul 09 02:46:49.482996 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:49.482606    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5fc988df-f28sl" event={"ID":"9204e949-bc8e-4d39-96da-1dfd97a63398","Type":"ContainerDied","Data":"8d377fce6dc58b1547e8050a25a8ff945d19162e514440db1448345e242239f4"}
Jul 09 02:46:49.482996 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:49.482663    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5fc988df-f28sl" event={"ID":"9204e949-bc8e-4d39-96da-1dfd97a63398","Type":"ContainerDied","Data":"fbe9816a04fbf264e3a198bde44113e9a3562435370a0823f38a76d0814b1860"}
Jul 09 02:46:49.482996 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:49.482684    2578 scope.go:117] "RemoveContainer" containerID="8d377fce6dc58b1547e8050a25a8ff945d19162e514440db1448345e242239f4"
Jul 09 02:46:49.491843 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:49.491827    2578 scope.go:117] "RemoveContainer" containerID="8d377fce6dc58b1547e8050a25a8ff945d19162e514440db1448345e242239f4"
Jul 09 02:46:49.492094 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:46:49.492074    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d377fce6dc58b1547e8050a25a8ff945d19162e514440db1448345e242239f4\": container with ID starting with 8d377fce6dc58b1547e8050a25a8ff945d19162e514440db1448345e242239f4 not found: ID does not exist" containerID="8d377fce6dc58b1547e8050a25a8ff945d19162e514440db1448345e242239f4"
Jul 09 02:46:49.492155 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:49.492107    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d377fce6dc58b1547e8050a25a8ff945d19162e514440db1448345e242239f4"} err="failed to get container status \"8d377fce6dc58b1547e8050a25a8ff945d19162e514440db1448345e242239f4\": rpc error: code = NotFound desc = could not find container \"8d377fce6dc58b1547e8050a25a8ff945d19162e514440db1448345e242239f4\": container with ID starting with 8d377fce6dc58b1547e8050a25a8ff945d19162e514440db1448345e242239f4 not found: ID does not exist"
Jul 09 02:46:49.502654 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:49.502629    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c5fc988df-f28sl"]
Jul 09 02:46:49.506511 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:49.506487    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5c5fc988df-f28sl"]
Jul 09 02:46:50.117449 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.117417    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9204e949-bc8e-4d39-96da-1dfd97a63398" path="/var/lib/kubelet/pods/9204e949-bc8e-4d39-96da-1dfd97a63398/volumes"
Jul 09 02:46:50.254379 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.254352    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c5fc988df-dkq9d"]
Jul 09 02:46:50.254751 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.254739    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9204e949-bc8e-4d39-96da-1dfd97a63398" containerName="manager"
Jul 09 02:46:50.254806 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.254752    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9204e949-bc8e-4d39-96da-1dfd97a63398" containerName="manager"
Jul 09 02:46:50.254806 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.254767    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7" containerName="cleanup"
Jul 09 02:46:50.254806 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.254773    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7" containerName="cleanup"
Jul 09 02:46:50.254908 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.254839    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba27bd7f-cf5f-4a2f-a51e-00ab7cb35cf7" containerName="cleanup"
Jul 09 02:46:50.254908 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.254848    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9204e949-bc8e-4d39-96da-1dfd97a63398" containerName="manager"
Jul 09 02:46:50.259539 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.259510    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c5fc988df-dkq9d"
Jul 09 02:46:50.262268 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.262244    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 09 02:46:50.262475 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.262460    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-v9sxg\""
Jul 09 02:46:50.262538 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.262518    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 09 02:46:50.266404 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.266381    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c5fc988df-dkq9d"]
Jul 09 02:46:50.345354 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.345324    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khp7w\" (UniqueName: \"kubernetes.io/projected/42b4081d-2fb4-422a-acc5-2e707eae8a94-kube-api-access-khp7w\") pod \"maas-controller-5c5fc988df-dkq9d\" (UID: \"42b4081d-2fb4-422a-acc5-2e707eae8a94\") " pod="opendatahub/maas-controller-5c5fc988df-dkq9d"
Jul 09 02:46:50.345485 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.345360    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42b4081d-2fb4-422a-acc5-2e707eae8a94-webhook-cert\") pod \"maas-controller-5c5fc988df-dkq9d\" (UID: \"42b4081d-2fb4-422a-acc5-2e707eae8a94\") " pod="opendatahub/maas-controller-5c5fc988df-dkq9d"
Jul 09 02:46:50.446309 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.446243    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khp7w\" (UniqueName: \"kubernetes.io/projected/42b4081d-2fb4-422a-acc5-2e707eae8a94-kube-api-access-khp7w\") pod \"maas-controller-5c5fc988df-dkq9d\" (UID: \"42b4081d-2fb4-422a-acc5-2e707eae8a94\") " pod="opendatahub/maas-controller-5c5fc988df-dkq9d"
Jul 09 02:46:50.446309 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.446276    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42b4081d-2fb4-422a-acc5-2e707eae8a94-webhook-cert\") pod \"maas-controller-5c5fc988df-dkq9d\" (UID: \"42b4081d-2fb4-422a-acc5-2e707eae8a94\") " pod="opendatahub/maas-controller-5c5fc988df-dkq9d"
Jul 09 02:46:50.448659 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.448641    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42b4081d-2fb4-422a-acc5-2e707eae8a94-webhook-cert\") pod \"maas-controller-5c5fc988df-dkq9d\" (UID: \"42b4081d-2fb4-422a-acc5-2e707eae8a94\") " pod="opendatahub/maas-controller-5c5fc988df-dkq9d"
Jul 09 02:46:50.453662 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.453642    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khp7w\" (UniqueName: \"kubernetes.io/projected/42b4081d-2fb4-422a-acc5-2e707eae8a94-kube-api-access-khp7w\") pod \"maas-controller-5c5fc988df-dkq9d\" (UID: \"42b4081d-2fb4-422a-acc5-2e707eae8a94\") " pod="opendatahub/maas-controller-5c5fc988df-dkq9d"
Jul 09 02:46:50.571193 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.571157    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c5fc988df-dkq9d"
Jul 09 02:46:50.701076 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.700933    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c5fc988df-dkq9d"]
Jul 09 02:46:50.703282 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:46:50.703249    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42b4081d_2fb4_422a_acc5_2e707eae8a94.slice/crio-52057ffa5957d87eb01f164dcd048d17a0c1ddea60aba5ae86d4810430d8aa86 WatchSource:0}: Error finding container 52057ffa5957d87eb01f164dcd048d17a0c1ddea60aba5ae86d4810430d8aa86: Status 404 returned error can't find the container with id 52057ffa5957d87eb01f164dcd048d17a0c1ddea60aba5ae86d4810430d8aa86
Jul 09 02:46:50.704615 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:50.704598    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 02:46:51.495287 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:51.495206    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5fc988df-dkq9d" event={"ID":"42b4081d-2fb4-422a-acc5-2e707eae8a94","Type":"ContainerStarted","Data":"24f4b49509f1aa232a8757e678c03decf5e5c42b3eaa8fc712f6aae67723ad12"}
Jul 09 02:46:51.495287 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:51.495284    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c5fc988df-dkq9d" event={"ID":"42b4081d-2fb4-422a-acc5-2e707eae8a94","Type":"ContainerStarted","Data":"52057ffa5957d87eb01f164dcd048d17a0c1ddea60aba5ae86d4810430d8aa86"}
Jul 09 02:46:51.495489 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:51.495367    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c5fc988df-dkq9d"
Jul 09 02:46:51.511308 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:46:51.511261    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c5fc988df-dkq9d" podStartSLOduration=1.071604885 podStartE2EDuration="1.511249508s" totalImagesPullingTime="439.644623ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:46:51.509738403 +0000 UTC m=+725.903541819" watchObservedRunningTime="2026-07-09 02:46:51.511249508 +0000 UTC m=+725.905052921"
Jul 09 02:47:02.505689 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:47:02.505658    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c5fc988df-dkq9d"
Jul 09 02:51:35.232669 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:35.232633    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"]
Jul 09 02:51:35.236026 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:35.232950    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs" podUID="4d6dc778-706e-42e2-9f42-b9bff65d93e0" containerName="manager" containerID="cri-o://91a44ab22a3105368ad418c4a649679b7808046140e8c48319963242268e5a3b" gracePeriod=10
Jul 09 02:51:37.786912 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:37.786883    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"
Jul 09 02:51:37.905914 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:37.905833    2578 generic.go:356] "Generic (PLEG): container finished" podID="4d6dc778-706e-42e2-9f42-b9bff65d93e0" containerID="91a44ab22a3105368ad418c4a649679b7808046140e8c48319963242268e5a3b" exitCode=0
Jul 09 02:51:37.905914 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:37.905884    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"
Jul 09 02:51:37.905914 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:37.905911    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs" event={"ID":"4d6dc778-706e-42e2-9f42-b9bff65d93e0","Type":"ContainerDied","Data":"91a44ab22a3105368ad418c4a649679b7808046140e8c48319963242268e5a3b"}
Jul 09 02:51:37.906163 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:37.905952    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs" event={"ID":"4d6dc778-706e-42e2-9f42-b9bff65d93e0","Type":"ContainerDied","Data":"a5a5052f96a00ef73568dcdd72817985640ee4c86fe21c9b5174daeca3da5748"}
Jul 09 02:51:37.906163 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:37.905973    2578 scope.go:117] "RemoveContainer" containerID="91a44ab22a3105368ad418c4a649679b7808046140e8c48319963242268e5a3b"
Jul 09 02:51:37.916011 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:37.915985    2578 scope.go:117] "RemoveContainer" containerID="91a44ab22a3105368ad418c4a649679b7808046140e8c48319963242268e5a3b"
Jul 09 02:51:37.916264 ip-10-0-129-236 kubenswrapper[2578]: E0709 02:51:37.916236    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91a44ab22a3105368ad418c4a649679b7808046140e8c48319963242268e5a3b\": container with ID starting with 91a44ab22a3105368ad418c4a649679b7808046140e8c48319963242268e5a3b not found: ID does not exist" containerID="91a44ab22a3105368ad418c4a649679b7808046140e8c48319963242268e5a3b"
Jul 09 02:51:37.916352 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:37.916274    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91a44ab22a3105368ad418c4a649679b7808046140e8c48319963242268e5a3b"} err="failed to get container status \"91a44ab22a3105368ad418c4a649679b7808046140e8c48319963242268e5a3b\": rpc error: code = NotFound desc = could not find container \"91a44ab22a3105368ad418c4a649679b7808046140e8c48319963242268e5a3b\": container with ID starting with 91a44ab22a3105368ad418c4a649679b7808046140e8c48319963242268e5a3b not found: ID does not exist"
Jul 09 02:51:37.921589 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:37.921572    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d6dc778-706e-42e2-9f42-b9bff65d93e0-extensions-socket-volume\") pod \"4d6dc778-706e-42e2-9f42-b9bff65d93e0\" (UID: \"4d6dc778-706e-42e2-9f42-b9bff65d93e0\") "
Jul 09 02:51:37.921645 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:37.921626    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nf9kj\" (UniqueName: \"kubernetes.io/projected/4d6dc778-706e-42e2-9f42-b9bff65d93e0-kube-api-access-nf9kj\") pod \"4d6dc778-706e-42e2-9f42-b9bff65d93e0\" (UID: \"4d6dc778-706e-42e2-9f42-b9bff65d93e0\") "
Jul 09 02:51:37.921905 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:37.921886    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d6dc778-706e-42e2-9f42-b9bff65d93e0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4d6dc778-706e-42e2-9f42-b9bff65d93e0" (UID: "4d6dc778-706e-42e2-9f42-b9bff65d93e0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 02:51:37.923548 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:37.923521    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d6dc778-706e-42e2-9f42-b9bff65d93e0-kube-api-access-nf9kj" (OuterVolumeSpecName: "kube-api-access-nf9kj") pod "4d6dc778-706e-42e2-9f42-b9bff65d93e0" (UID: "4d6dc778-706e-42e2-9f42-b9bff65d93e0"). InnerVolumeSpecName "kube-api-access-nf9kj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 02:51:38.023124 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:38.023096    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d6dc778-706e-42e2-9f42-b9bff65d93e0-extensions-socket-volume\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:51:38.023124 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:38.023117    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nf9kj\" (UniqueName: \"kubernetes.io/projected/4d6dc778-706e-42e2-9f42-b9bff65d93e0-kube-api-access-nf9kj\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 02:51:38.222885 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:38.222816    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"]
Jul 09 02:51:38.226599 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:38.226570    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57jfs"]
Jul 09 02:51:40.123129 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:51:40.123092    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d6dc778-706e-42e2-9f42-b9bff65d93e0" path="/var/lib/kubelet/pods/4d6dc778-706e-42e2-9f42-b9bff65d93e0/volumes"
Jul 09 02:52:36.375461 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:36.375427    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g496q"]
Jul 09 02:52:36.375937 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:36.375828    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d6dc778-706e-42e2-9f42-b9bff65d93e0" containerName="manager"
Jul 09 02:52:36.375937 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:36.375840    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d6dc778-706e-42e2-9f42-b9bff65d93e0" containerName="manager"
Jul 09 02:52:36.375937 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:36.375901    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d6dc778-706e-42e2-9f42-b9bff65d93e0" containerName="manager"
Jul 09 02:52:36.378984 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:36.378968    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g496q"
Jul 09 02:52:36.381241 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:36.381219    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2cq5r\""
Jul 09 02:52:36.389699 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:36.389675    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g496q"]
Jul 09 02:52:36.517062 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:36.517026    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7dfz\" (UniqueName: \"kubernetes.io/projected/c9711e01-33c9-4967-8001-bbe3c904f253-kube-api-access-k7dfz\") pod \"kuadrant-operator-controller-manager-86878c769f-g496q\" (UID: \"c9711e01-33c9-4967-8001-bbe3c904f253\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g496q"
Jul 09 02:52:36.517240 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:36.517151    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9711e01-33c9-4967-8001-bbe3c904f253-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g496q\" (UID: \"c9711e01-33c9-4967-8001-bbe3c904f253\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g496q"
Jul 09 02:52:36.617840 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:36.617804    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9711e01-33c9-4967-8001-bbe3c904f253-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g496q\" (UID: \"c9711e01-33c9-4967-8001-bbe3c904f253\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g496q"
Jul 09 02:52:36.617840 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:36.617854    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7dfz\" (UniqueName: \"kubernetes.io/projected/c9711e01-33c9-4967-8001-bbe3c904f253-kube-api-access-k7dfz\") pod \"kuadrant-operator-controller-manager-86878c769f-g496q\" (UID: \"c9711e01-33c9-4967-8001-bbe3c904f253\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g496q"
Jul 09 02:52:36.618277 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:36.618251    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9711e01-33c9-4967-8001-bbe3c904f253-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g496q\" (UID: \"c9711e01-33c9-4967-8001-bbe3c904f253\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g496q"
Jul 09 02:52:36.630740 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:36.630672    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7dfz\" (UniqueName: \"kubernetes.io/projected/c9711e01-33c9-4967-8001-bbe3c904f253-kube-api-access-k7dfz\") pod \"kuadrant-operator-controller-manager-86878c769f-g496q\" (UID: \"c9711e01-33c9-4967-8001-bbe3c904f253\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g496q"
Jul 09 02:52:36.689713 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:36.689678    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g496q"
Jul 09 02:52:36.830852 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:36.830813    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g496q"]
Jul 09 02:52:36.834496 ip-10-0-129-236 kubenswrapper[2578]: W0709 02:52:36.834460    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9711e01_33c9_4967_8001_bbe3c904f253.slice/crio-87744737a292268d5a71c6582602dc46c268fa3b1f48a7b2f040c1960e955ec2 WatchSource:0}: Error finding container 87744737a292268d5a71c6582602dc46c268fa3b1f48a7b2f040c1960e955ec2: Status 404 returned error can't find the container with id 87744737a292268d5a71c6582602dc46c268fa3b1f48a7b2f040c1960e955ec2
Jul 09 02:52:37.199427 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:37.199339    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g496q" event={"ID":"c9711e01-33c9-4967-8001-bbe3c904f253","Type":"ContainerStarted","Data":"f2a5c2386e52d807c3c16677d753b57d08244a95edeebd79b9c68e19cd0f9d83"}
Jul 09 02:52:37.199427 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:37.199396    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g496q" event={"ID":"c9711e01-33c9-4967-8001-bbe3c904f253","Type":"ContainerStarted","Data":"87744737a292268d5a71c6582602dc46c268fa3b1f48a7b2f040c1960e955ec2"}
Jul 09 02:52:37.199427 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:37.199427    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g496q"
Jul 09 02:52:37.219090 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:37.219043    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g496q" podStartSLOduration=1.219028862 podStartE2EDuration="1.219028862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 02:52:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 02:52:37.216730569 +0000 UTC m=+1071.610533982" watchObservedRunningTime="2026-07-09 02:52:37.219028862 +0000 UTC m=+1071.612832275"
Jul 09 02:52:48.206211 ip-10-0-129-236 kubenswrapper[2578]: I0709 02:52:48.206174    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g496q"
Jul 09 03:00:00.137463 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:00:00.137429    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g"]
Jul 09 03:00:00.147615 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:00:00.147585    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g"
Jul 09 03:00:00.156868 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:00:00.156836    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g"]
Jul 09 03:00:00.299421 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:00:00.299389    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctlnd\" (UniqueName: \"kubernetes.io/projected/7f332565-dd60-422c-8664-191fdc37ca37-kube-api-access-ctlnd\") pod \"maas-api-key-cleanup-29726100-6wl5g\" (UID: \"7f332565-dd60-422c-8664-191fdc37ca37\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g"
Jul 09 03:00:00.400885 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:00:00.400796    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctlnd\" (UniqueName: \"kubernetes.io/projected/7f332565-dd60-422c-8664-191fdc37ca37-kube-api-access-ctlnd\") pod \"maas-api-key-cleanup-29726100-6wl5g\" (UID: \"7f332565-dd60-422c-8664-191fdc37ca37\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g"
Jul 09 03:00:00.408289 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:00:00.408247    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctlnd\" (UniqueName: \"kubernetes.io/projected/7f332565-dd60-422c-8664-191fdc37ca37-kube-api-access-ctlnd\") pod \"maas-api-key-cleanup-29726100-6wl5g\" (UID: \"7f332565-dd60-422c-8664-191fdc37ca37\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g"
Jul 09 03:00:00.458791 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:00:00.458763    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g"
Jul 09 03:00:00.591653 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:00:00.591627    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g"]
Jul 09 03:00:01.381932 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:00:01.381892    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g" event={"ID":"7f332565-dd60-422c-8664-191fdc37ca37","Type":"ContainerStarted","Data":"359bbbde51d29a807c68b6aac5131b38a388f0a539981a014fa38704ea75a8d3"}
Jul 09 03:00:01.382345 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:00:01.381955    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g" event={"ID":"7f332565-dd60-422c-8664-191fdc37ca37","Type":"ContainerStarted","Data":"6d182a06ac0a311d8abf08c2b09f07bef470fcc8952b26371d478939b8e44729"}
Jul 09 03:00:01.395448 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:00:01.395368    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g" podStartSLOduration=1.395348219 podStartE2EDuration="1.395348219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:00:01.395100215 +0000 UTC m=+1515.788903644" watchObservedRunningTime="2026-07-09 03:00:01.395348219 +0000 UTC m=+1515.789151632"
Jul 09 03:01:00.009491 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:01:00.009452    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g"]
Jul 09 03:01:00.009955 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:01:00.009667    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g" podUID="7f332565-dd60-422c-8664-191fdc37ca37" containerName="cleanup" containerID="cri-o://359bbbde51d29a807c68b6aac5131b38a388f0a539981a014fa38704ea75a8d3" gracePeriod=30
Jul 09 03:01:30.193821 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:01:30.193798    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g"
Jul 09 03:01:30.312873 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:01:30.312777    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctlnd\" (UniqueName: \"kubernetes.io/projected/7f332565-dd60-422c-8664-191fdc37ca37-kube-api-access-ctlnd\") pod \"7f332565-dd60-422c-8664-191fdc37ca37\" (UID: \"7f332565-dd60-422c-8664-191fdc37ca37\") "
Jul 09 03:01:30.314775 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:01:30.314750    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f332565-dd60-422c-8664-191fdc37ca37-kube-api-access-ctlnd" (OuterVolumeSpecName: "kube-api-access-ctlnd") pod "7f332565-dd60-422c-8664-191fdc37ca37" (UID: "7f332565-dd60-422c-8664-191fdc37ca37"). InnerVolumeSpecName "kube-api-access-ctlnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:01:30.413988 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:01:30.413951    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctlnd\" (UniqueName: \"kubernetes.io/projected/7f332565-dd60-422c-8664-191fdc37ca37-kube-api-access-ctlnd\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:01:30.816824 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:01:30.816791    2578 generic.go:356] "Generic (PLEG): container finished" podID="7f332565-dd60-422c-8664-191fdc37ca37" containerID="359bbbde51d29a807c68b6aac5131b38a388f0a539981a014fa38704ea75a8d3" exitCode=137
Jul 09 03:01:30.816991 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:01:30.816835    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g"
Jul 09 03:01:30.816991 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:01:30.816877    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g" event={"ID":"7f332565-dd60-422c-8664-191fdc37ca37","Type":"ContainerDied","Data":"359bbbde51d29a807c68b6aac5131b38a388f0a539981a014fa38704ea75a8d3"}
Jul 09 03:01:30.816991 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:01:30.816927    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g" event={"ID":"7f332565-dd60-422c-8664-191fdc37ca37","Type":"ContainerDied","Data":"6d182a06ac0a311d8abf08c2b09f07bef470fcc8952b26371d478939b8e44729"}
Jul 09 03:01:30.816991 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:01:30.816949    2578 scope.go:117] "RemoveContainer" containerID="359bbbde51d29a807c68b6aac5131b38a388f0a539981a014fa38704ea75a8d3"
Jul 09 03:01:30.826142 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:01:30.826126    2578 scope.go:117] "RemoveContainer" containerID="359bbbde51d29a807c68b6aac5131b38a388f0a539981a014fa38704ea75a8d3"
Jul 09 03:01:30.826425 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:01:30.826403    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"359bbbde51d29a807c68b6aac5131b38a388f0a539981a014fa38704ea75a8d3\": container with ID starting with 359bbbde51d29a807c68b6aac5131b38a388f0a539981a014fa38704ea75a8d3 not found: ID does not exist" containerID="359bbbde51d29a807c68b6aac5131b38a388f0a539981a014fa38704ea75a8d3"
Jul 09 03:01:30.826494 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:01:30.826434    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"359bbbde51d29a807c68b6aac5131b38a388f0a539981a014fa38704ea75a8d3"} err="failed to get container status \"359bbbde51d29a807c68b6aac5131b38a388f0a539981a014fa38704ea75a8d3\": rpc error: code = NotFound desc = could not find container \"359bbbde51d29a807c68b6aac5131b38a388f0a539981a014fa38704ea75a8d3\": container with ID starting with 359bbbde51d29a807c68b6aac5131b38a388f0a539981a014fa38704ea75a8d3 not found: ID does not exist"
Jul 09 03:01:30.836713 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:01:30.836665    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g"]
Jul 09 03:01:30.838359 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:01:30.838336    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726100-6wl5g"]
Jul 09 03:01:32.117490 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:01:32.117455    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f332565-dd60-422c-8664-191fdc37ca37" path="/var/lib/kubelet/pods/7f332565-dd60-422c-8664-191fdc37ca37/volumes"
Jul 09 03:05:09.786928 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:09.786893    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"]
Jul 09 03:05:09.787473 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:09.787310    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f332565-dd60-422c-8664-191fdc37ca37" containerName="cleanup"
Jul 09 03:05:09.787473 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:09.787341    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f332565-dd60-422c-8664-191fdc37ca37" containerName="cleanup"
Jul 09 03:05:09.787473 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:09.787441    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f332565-dd60-422c-8664-191fdc37ca37" containerName="cleanup"
Jul 09 03:05:09.790718 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:09.790699    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"
Jul 09 03:05:09.792803 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:09.792779    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-44d03cda-serving-cert\""
Jul 09 03:05:09.800572 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:09.800550    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"]
Jul 09 03:05:09.894364 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:09.894306    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-openshift-service-ca\") pod \"maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5\" (UID: \"fc6ca40f-eaba-49c5-b405-f6eddc6fab62\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"
Jul 09 03:05:09.894630 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:09.894609    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-maas-api-tls\") pod \"maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5\" (UID: \"fc6ca40f-eaba-49c5-b405-f6eddc6fab62\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"
Jul 09 03:05:09.894837 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:09.894823    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfmjp\" (UniqueName: \"kubernetes.io/projected/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-kube-api-access-bfmjp\") pod \"maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5\" (UID: \"fc6ca40f-eaba-49c5-b405-f6eddc6fab62\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"
Jul 09 03:05:09.995896 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:09.995854    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfmjp\" (UniqueName: \"kubernetes.io/projected/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-kube-api-access-bfmjp\") pod \"maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5\" (UID: \"fc6ca40f-eaba-49c5-b405-f6eddc6fab62\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"
Jul 09 03:05:09.996079 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:09.995931    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-openshift-service-ca\") pod \"maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5\" (UID: \"fc6ca40f-eaba-49c5-b405-f6eddc6fab62\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"
Jul 09 03:05:09.996079 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:09.995969    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-maas-api-tls\") pod \"maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5\" (UID: \"fc6ca40f-eaba-49c5-b405-f6eddc6fab62\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"
Jul 09 03:05:09.996196 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:05:09.996102    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-serving-cert: secret "maas-api-e2e-ait-44d03cda-serving-cert" not found
Jul 09 03:05:09.996196 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:05:09.996151    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-maas-api-tls podName:fc6ca40f-eaba-49c5-b405-f6eddc6fab62 nodeName:}" failed. No retries permitted until 2026-07-09 03:05:10.496135559 +0000 UTC m=+1824.889938951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-maas-api-tls") pod "maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5" (UID: "fc6ca40f-eaba-49c5-b405-f6eddc6fab62") : secret "maas-api-e2e-ait-44d03cda-serving-cert" not found
Jul 09 03:05:09.996651 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:09.996630    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-openshift-service-ca\") pod \"maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5\" (UID: \"fc6ca40f-eaba-49c5-b405-f6eddc6fab62\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"
Jul 09 03:05:10.003980 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:10.003955    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfmjp\" (UniqueName: \"kubernetes.io/projected/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-kube-api-access-bfmjp\") pod \"maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5\" (UID: \"fc6ca40f-eaba-49c5-b405-f6eddc6fab62\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"
Jul 09 03:05:10.500373 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:10.500335    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-maas-api-tls\") pod \"maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5\" (UID: \"fc6ca40f-eaba-49c5-b405-f6eddc6fab62\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"
Jul 09 03:05:10.502785 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:10.502754    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-maas-api-tls\") pod \"maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5\" (UID: \"fc6ca40f-eaba-49c5-b405-f6eddc6fab62\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"
Jul 09 03:05:10.702342 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:10.702271    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"
Jul 09 03:05:10.837474 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:10.837446    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"]
Jul 09 03:05:10.839507 ip-10-0-129-236 kubenswrapper[2578]: W0709 03:05:10.839480    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc6ca40f_eaba_49c5_b405_f6eddc6fab62.slice/crio-590722ff8e26d1459f9fd81981fa6c2cfc908fda1974101bdd8ddfac715d206e WatchSource:0}: Error finding container 590722ff8e26d1459f9fd81981fa6c2cfc908fda1974101bdd8ddfac715d206e: Status 404 returned error can't find the container with id 590722ff8e26d1459f9fd81981fa6c2cfc908fda1974101bdd8ddfac715d206e
Jul 09 03:05:10.840862 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:10.840844    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 03:05:10.871647 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:10.871615    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5" event={"ID":"fc6ca40f-eaba-49c5-b405-f6eddc6fab62","Type":"ContainerStarted","Data":"590722ff8e26d1459f9fd81981fa6c2cfc908fda1974101bdd8ddfac715d206e"}
Jul 09 03:05:11.877952 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:11.877916    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5" event={"ID":"fc6ca40f-eaba-49c5-b405-f6eddc6fab62","Type":"ContainerStarted","Data":"5a01d306fe7aed21473cf546ec912c907271408f131047f30287a0ebad9df35a"}
Jul 09 03:05:11.878357 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:11.878085    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"
Jul 09 03:05:11.894832 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:11.894781    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5" podStartSLOduration=2.366700244 podStartE2EDuration="2.894765959s" totalImagesPullingTime="528.065715ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:05:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:05:11.892632754 +0000 UTC m=+1826.286436161" watchObservedRunningTime="2026-07-09 03:05:11.894765959 +0000 UTC m=+1826.288569372"
Jul 09 03:05:14.439529 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.439490    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"]
Jul 09 03:05:14.440373 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.440296    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5" podUID="fc6ca40f-eaba-49c5-b405-f6eddc6fab62" containerName="maas-api" containerID="cri-o://5a01d306fe7aed21473cf546ec912c907271408f131047f30287a0ebad9df35a" gracePeriod=30
Jul 09 03:05:14.692247 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.692192    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"
Jul 09 03:05:14.840415 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.840377    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfmjp\" (UniqueName: \"kubernetes.io/projected/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-kube-api-access-bfmjp\") pod \"fc6ca40f-eaba-49c5-b405-f6eddc6fab62\" (UID: \"fc6ca40f-eaba-49c5-b405-f6eddc6fab62\") "
Jul 09 03:05:14.840710 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.840457    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-maas-api-tls\") pod \"fc6ca40f-eaba-49c5-b405-f6eddc6fab62\" (UID: \"fc6ca40f-eaba-49c5-b405-f6eddc6fab62\") "
Jul 09 03:05:14.840710 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.840480    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-openshift-service-ca\") pod \"fc6ca40f-eaba-49c5-b405-f6eddc6fab62\" (UID: \"fc6ca40f-eaba-49c5-b405-f6eddc6fab62\") "
Jul 09 03:05:14.840885 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.840859    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fc6ca40f-eaba-49c5-b405-f6eddc6fab62" (UID: "fc6ca40f-eaba-49c5-b405-f6eddc6fab62"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 03:05:14.842770 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.842734    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fc6ca40f-eaba-49c5-b405-f6eddc6fab62" (UID: "fc6ca40f-eaba-49c5-b405-f6eddc6fab62"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 03:05:14.842770 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.842749    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-kube-api-access-bfmjp" (OuterVolumeSpecName: "kube-api-access-bfmjp") pod "fc6ca40f-eaba-49c5-b405-f6eddc6fab62" (UID: "fc6ca40f-eaba-49c5-b405-f6eddc6fab62"). InnerVolumeSpecName "kube-api-access-bfmjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:05:14.895480 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.895448    2578 generic.go:356] "Generic (PLEG): container finished" podID="fc6ca40f-eaba-49c5-b405-f6eddc6fab62" containerID="5a01d306fe7aed21473cf546ec912c907271408f131047f30287a0ebad9df35a" exitCode=0
Jul 09 03:05:14.895610 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.895496    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"
Jul 09 03:05:14.895610 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.895537    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5" event={"ID":"fc6ca40f-eaba-49c5-b405-f6eddc6fab62","Type":"ContainerDied","Data":"5a01d306fe7aed21473cf546ec912c907271408f131047f30287a0ebad9df35a"}
Jul 09 03:05:14.895610 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.895593    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5" event={"ID":"fc6ca40f-eaba-49c5-b405-f6eddc6fab62","Type":"ContainerDied","Data":"590722ff8e26d1459f9fd81981fa6c2cfc908fda1974101bdd8ddfac715d206e"}
Jul 09 03:05:14.895763 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.895622    2578 scope.go:117] "RemoveContainer" containerID="5a01d306fe7aed21473cf546ec912c907271408f131047f30287a0ebad9df35a"
Jul 09 03:05:14.904355 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.904304    2578 scope.go:117] "RemoveContainer" containerID="5a01d306fe7aed21473cf546ec912c907271408f131047f30287a0ebad9df35a"
Jul 09 03:05:14.904616 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:05:14.904599    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a01d306fe7aed21473cf546ec912c907271408f131047f30287a0ebad9df35a\": container with ID starting with 5a01d306fe7aed21473cf546ec912c907271408f131047f30287a0ebad9df35a not found: ID does not exist" containerID="5a01d306fe7aed21473cf546ec912c907271408f131047f30287a0ebad9df35a"
Jul 09 03:05:14.904668 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.904625    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a01d306fe7aed21473cf546ec912c907271408f131047f30287a0ebad9df35a"} err="failed to get container status \"5a01d306fe7aed21473cf546ec912c907271408f131047f30287a0ebad9df35a\": rpc error: code = NotFound desc = could not find container \"5a01d306fe7aed21473cf546ec912c907271408f131047f30287a0ebad9df35a\": container with ID starting with 5a01d306fe7aed21473cf546ec912c907271408f131047f30287a0ebad9df35a not found: ID does not exist"
Jul 09 03:05:14.915442 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.915415    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"]
Jul 09 03:05:14.920879 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.920860    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-44d03cda-c58fc46d9-rmzf5"]
Jul 09 03:05:14.941199 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.941181    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-maas-api-tls\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:05:14.941199 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.941199    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-openshift-service-ca\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:05:14.941356 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:14.941211    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfmjp\" (UniqueName: \"kubernetes.io/projected/fc6ca40f-eaba-49c5-b405-f6eddc6fab62-kube-api-access-bfmjp\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:05:16.117981 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:16.117947    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc6ca40f-eaba-49c5-b405-f6eddc6fab62" path="/var/lib/kubelet/pods/fc6ca40f-eaba-49c5-b405-f6eddc6fab62/volumes"
Jul 09 03:05:34.996186 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:34.996151    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"]
Jul 09 03:05:34.996963 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:34.996940    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc6ca40f-eaba-49c5-b405-f6eddc6fab62" containerName="maas-api"
Jul 09 03:05:34.997054 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:34.996966    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc6ca40f-eaba-49c5-b405-f6eddc6fab62" containerName="maas-api"
Jul 09 03:05:34.997139 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:34.997125    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc6ca40f-eaba-49c5-b405-f6eddc6fab62" containerName="maas-api"
Jul 09 03:05:35.000477 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:35.000455    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"
Jul 09 03:05:35.005048 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:35.005030    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-35ef5376-serving-cert\""
Jul 09 03:05:35.009970 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:35.009947    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"]
Jul 09 03:05:35.121692 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:35.121658    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e878192e-c61f-40fe-b463-e3a2c8a88002-maas-api-tls\") pod \"maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5\" (UID: \"e878192e-c61f-40fe-b463-e3a2c8a88002\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"
Jul 09 03:05:35.121865 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:35.121704    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x296l\" (UniqueName: \"kubernetes.io/projected/e878192e-c61f-40fe-b463-e3a2c8a88002-kube-api-access-x296l\") pod \"maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5\" (UID: \"e878192e-c61f-40fe-b463-e3a2c8a88002\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"
Jul 09 03:05:35.121865 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:35.121831    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e878192e-c61f-40fe-b463-e3a2c8a88002-openshift-service-ca\") pod \"maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5\" (UID: \"e878192e-c61f-40fe-b463-e3a2c8a88002\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"
Jul 09 03:05:35.222828 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:35.222792    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e878192e-c61f-40fe-b463-e3a2c8a88002-openshift-service-ca\") pod \"maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5\" (UID: \"e878192e-c61f-40fe-b463-e3a2c8a88002\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"
Jul 09 03:05:35.223000 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:35.222842    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e878192e-c61f-40fe-b463-e3a2c8a88002-maas-api-tls\") pod \"maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5\" (UID: \"e878192e-c61f-40fe-b463-e3a2c8a88002\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"
Jul 09 03:05:35.223000 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:35.222871    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x296l\" (UniqueName: \"kubernetes.io/projected/e878192e-c61f-40fe-b463-e3a2c8a88002-kube-api-access-x296l\") pod \"maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5\" (UID: \"e878192e-c61f-40fe-b463-e3a2c8a88002\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"
Jul 09 03:05:35.223078 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:05:35.222998    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-serving-cert: secret "maas-api-e2e-derive-35ef5376-serving-cert" not found
Jul 09 03:05:35.223078 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:05:35.223060    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e878192e-c61f-40fe-b463-e3a2c8a88002-maas-api-tls podName:e878192e-c61f-40fe-b463-e3a2c8a88002 nodeName:}" failed. No retries permitted until 2026-07-09 03:05:35.723044329 +0000 UTC m=+1850.116847723 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e878192e-c61f-40fe-b463-e3a2c8a88002-maas-api-tls") pod "maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5" (UID: "e878192e-c61f-40fe-b463-e3a2c8a88002") : secret "maas-api-e2e-derive-35ef5376-serving-cert" not found
Jul 09 03:05:35.223427 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:35.223409    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e878192e-c61f-40fe-b463-e3a2c8a88002-openshift-service-ca\") pod \"maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5\" (UID: \"e878192e-c61f-40fe-b463-e3a2c8a88002\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"
Jul 09 03:05:35.231399 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:35.231374    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x296l\" (UniqueName: \"kubernetes.io/projected/e878192e-c61f-40fe-b463-e3a2c8a88002-kube-api-access-x296l\") pod \"maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5\" (UID: \"e878192e-c61f-40fe-b463-e3a2c8a88002\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"
Jul 09 03:05:35.727479 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:35.727449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e878192e-c61f-40fe-b463-e3a2c8a88002-maas-api-tls\") pod \"maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5\" (UID: \"e878192e-c61f-40fe-b463-e3a2c8a88002\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"
Jul 09 03:05:35.729894 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:35.729870    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e878192e-c61f-40fe-b463-e3a2c8a88002-maas-api-tls\") pod \"maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5\" (UID: \"e878192e-c61f-40fe-b463-e3a2c8a88002\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"
Jul 09 03:05:35.912713 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:35.912668    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"
Jul 09 03:05:36.050445 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:36.050416    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"]
Jul 09 03:05:36.053187 ip-10-0-129-236 kubenswrapper[2578]: W0709 03:05:36.053157    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode878192e_c61f_40fe_b463_e3a2c8a88002.slice/crio-5809d741043352a2b35d811d0cf2570a023a0188601be37197721512ec0c404c WatchSource:0}: Error finding container 5809d741043352a2b35d811d0cf2570a023a0188601be37197721512ec0c404c: Status 404 returned error can't find the container with id 5809d741043352a2b35d811d0cf2570a023a0188601be37197721512ec0c404c
Jul 09 03:05:37.010109 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:37.010013    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5" event={"ID":"e878192e-c61f-40fe-b463-e3a2c8a88002","Type":"ContainerStarted","Data":"6ee526530fa92b47e6d350f868c5410d7c02cf414189d9351e40dff1f06cf528"}
Jul 09 03:05:37.010109 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:37.010064    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5" event={"ID":"e878192e-c61f-40fe-b463-e3a2c8a88002","Type":"ContainerStarted","Data":"5809d741043352a2b35d811d0cf2570a023a0188601be37197721512ec0c404c"}
Jul 09 03:05:37.010357 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:37.010109    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"
Jul 09 03:05:37.025904 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:37.025857    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5" podStartSLOduration=2.470522386 podStartE2EDuration="3.025843485s" totalImagesPullingTime="555.321099ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:05:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:05:37.024035785 +0000 UTC m=+1851.417839198" watchObservedRunningTime="2026-07-09 03:05:37.025843485 +0000 UTC m=+1851.419646898"
Jul 09 03:05:39.259225 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:39.259194    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"]
Jul 09 03:05:39.259707 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:39.259448    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5" podUID="e878192e-c61f-40fe-b463-e3a2c8a88002" containerName="maas-api" containerID="cri-o://6ee526530fa92b47e6d350f868c5410d7c02cf414189d9351e40dff1f06cf528" gracePeriod=30
Jul 09 03:05:39.508423 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:39.508398    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"
Jul 09 03:05:39.567187 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:39.567112    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e878192e-c61f-40fe-b463-e3a2c8a88002-maas-api-tls\") pod \"e878192e-c61f-40fe-b463-e3a2c8a88002\" (UID: \"e878192e-c61f-40fe-b463-e3a2c8a88002\") "
Jul 09 03:05:39.567187 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:39.567180    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x296l\" (UniqueName: \"kubernetes.io/projected/e878192e-c61f-40fe-b463-e3a2c8a88002-kube-api-access-x296l\") pod \"e878192e-c61f-40fe-b463-e3a2c8a88002\" (UID: \"e878192e-c61f-40fe-b463-e3a2c8a88002\") "
Jul 09 03:05:39.567396 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:39.567262    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e878192e-c61f-40fe-b463-e3a2c8a88002-openshift-service-ca\") pod \"e878192e-c61f-40fe-b463-e3a2c8a88002\" (UID: \"e878192e-c61f-40fe-b463-e3a2c8a88002\") "
Jul 09 03:05:39.567638 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:39.567614    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e878192e-c61f-40fe-b463-e3a2c8a88002-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e878192e-c61f-40fe-b463-e3a2c8a88002" (UID: "e878192e-c61f-40fe-b463-e3a2c8a88002"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 03:05:39.569363 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:39.569338    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e878192e-c61f-40fe-b463-e3a2c8a88002-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e878192e-c61f-40fe-b463-e3a2c8a88002" (UID: "e878192e-c61f-40fe-b463-e3a2c8a88002"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 03:05:39.569708 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:39.569687    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e878192e-c61f-40fe-b463-e3a2c8a88002-kube-api-access-x296l" (OuterVolumeSpecName: "kube-api-access-x296l") pod "e878192e-c61f-40fe-b463-e3a2c8a88002" (UID: "e878192e-c61f-40fe-b463-e3a2c8a88002"). InnerVolumeSpecName "kube-api-access-x296l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:05:39.668030 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:39.667994    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e878192e-c61f-40fe-b463-e3a2c8a88002-maas-api-tls\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:05:39.668030 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:39.668024    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x296l\" (UniqueName: \"kubernetes.io/projected/e878192e-c61f-40fe-b463-e3a2c8a88002-kube-api-access-x296l\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:05:39.668228 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:39.668038    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e878192e-c61f-40fe-b463-e3a2c8a88002-openshift-service-ca\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:05:40.025353 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:40.025292    2578 generic.go:356] "Generic (PLEG): container finished" podID="e878192e-c61f-40fe-b463-e3a2c8a88002" containerID="6ee526530fa92b47e6d350f868c5410d7c02cf414189d9351e40dff1f06cf528" exitCode=0
Jul 09 03:05:40.025535 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:40.025366    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"
Jul 09 03:05:40.025535 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:40.025378    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5" event={"ID":"e878192e-c61f-40fe-b463-e3a2c8a88002","Type":"ContainerDied","Data":"6ee526530fa92b47e6d350f868c5410d7c02cf414189d9351e40dff1f06cf528"}
Jul 09 03:05:40.025535 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:40.025457    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5" event={"ID":"e878192e-c61f-40fe-b463-e3a2c8a88002","Type":"ContainerDied","Data":"5809d741043352a2b35d811d0cf2570a023a0188601be37197721512ec0c404c"}
Jul 09 03:05:40.025535 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:40.025480    2578 scope.go:117] "RemoveContainer" containerID="6ee526530fa92b47e6d350f868c5410d7c02cf414189d9351e40dff1f06cf528"
Jul 09 03:05:40.034996 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:40.034980    2578 scope.go:117] "RemoveContainer" containerID="6ee526530fa92b47e6d350f868c5410d7c02cf414189d9351e40dff1f06cf528"
Jul 09 03:05:40.035288 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:05:40.035268    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ee526530fa92b47e6d350f868c5410d7c02cf414189d9351e40dff1f06cf528\": container with ID starting with 6ee526530fa92b47e6d350f868c5410d7c02cf414189d9351e40dff1f06cf528 not found: ID does not exist" containerID="6ee526530fa92b47e6d350f868c5410d7c02cf414189d9351e40dff1f06cf528"
Jul 09 03:05:40.035372 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:40.035297    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ee526530fa92b47e6d350f868c5410d7c02cf414189d9351e40dff1f06cf528"} err="failed to get container status \"6ee526530fa92b47e6d350f868c5410d7c02cf414189d9351e40dff1f06cf528\": rpc error: code = NotFound desc = could not find container \"6ee526530fa92b47e6d350f868c5410d7c02cf414189d9351e40dff1f06cf528\": container with ID starting with 6ee526530fa92b47e6d350f868c5410d7c02cf414189d9351e40dff1f06cf528 not found: ID does not exist"
Jul 09 03:05:40.045198 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:40.045177    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"]
Jul 09 03:05:40.048946 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:40.048927    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-35ef5376-7f687f985f-4mlk5"]
Jul 09 03:05:40.117927 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:05:40.117894    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e878192e-c61f-40fe-b463-e3a2c8a88002" path="/var/lib/kubelet/pods/e878192e-c61f-40fe-b463-e3a2c8a88002/volumes"
Jul 09 03:11:00.802952 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:00.802911    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"]
Jul 09 03:11:00.803561 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:00.803402    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e878192e-c61f-40fe-b463-e3a2c8a88002" containerName="maas-api"
Jul 09 03:11:00.803561 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:00.803423    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e878192e-c61f-40fe-b463-e3a2c8a88002" containerName="maas-api"
Jul 09 03:11:00.803702 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:00.803604    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e878192e-c61f-40fe-b463-e3a2c8a88002" containerName="maas-api"
Jul 09 03:11:00.806711 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:00.806694    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"
Jul 09 03:11:00.808603 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:00.808584    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-28444007-serving-cert\""
Jul 09 03:11:00.817719 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:00.817694    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"]
Jul 09 03:11:00.909995 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:00.909969    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3891296-65af-4372-b960-6bf45fa64465-maas-api-tls\") pod \"maas-api-e2e-mt-28444007-7796df9dfd-b6zmq\" (UID: \"e3891296-65af-4372-b960-6bf45fa64465\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"
Jul 09 03:11:00.910143 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:00.910002    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm9mf\" (UniqueName: \"kubernetes.io/projected/e3891296-65af-4372-b960-6bf45fa64465-kube-api-access-cm9mf\") pod \"maas-api-e2e-mt-28444007-7796df9dfd-b6zmq\" (UID: \"e3891296-65af-4372-b960-6bf45fa64465\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"
Jul 09 03:11:00.910143 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:00.910029    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3891296-65af-4372-b960-6bf45fa64465-openshift-service-ca\") pod \"maas-api-e2e-mt-28444007-7796df9dfd-b6zmq\" (UID: \"e3891296-65af-4372-b960-6bf45fa64465\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"
Jul 09 03:11:01.011331 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:01.011293    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm9mf\" (UniqueName: \"kubernetes.io/projected/e3891296-65af-4372-b960-6bf45fa64465-kube-api-access-cm9mf\") pod \"maas-api-e2e-mt-28444007-7796df9dfd-b6zmq\" (UID: \"e3891296-65af-4372-b960-6bf45fa64465\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"
Jul 09 03:11:01.011456 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:01.011365    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3891296-65af-4372-b960-6bf45fa64465-openshift-service-ca\") pod \"maas-api-e2e-mt-28444007-7796df9dfd-b6zmq\" (UID: \"e3891296-65af-4372-b960-6bf45fa64465\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"
Jul 09 03:11:01.011521 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:01.011462    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3891296-65af-4372-b960-6bf45fa64465-maas-api-tls\") pod \"maas-api-e2e-mt-28444007-7796df9dfd-b6zmq\" (UID: \"e3891296-65af-4372-b960-6bf45fa64465\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"
Jul 09 03:11:01.011582 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:11:01.011568    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-28444007-serving-cert: secret "maas-api-e2e-mt-28444007-serving-cert" not found
Jul 09 03:11:01.011659 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:11:01.011647    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3891296-65af-4372-b960-6bf45fa64465-maas-api-tls podName:e3891296-65af-4372-b960-6bf45fa64465 nodeName:}" failed. No retries permitted until 2026-07-09 03:11:01.511626779 +0000 UTC m=+2175.905430169 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e3891296-65af-4372-b960-6bf45fa64465-maas-api-tls") pod "maas-api-e2e-mt-28444007-7796df9dfd-b6zmq" (UID: "e3891296-65af-4372-b960-6bf45fa64465") : secret "maas-api-e2e-mt-28444007-serving-cert" not found
Jul 09 03:11:01.012084 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:01.012066    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3891296-65af-4372-b960-6bf45fa64465-openshift-service-ca\") pod \"maas-api-e2e-mt-28444007-7796df9dfd-b6zmq\" (UID: \"e3891296-65af-4372-b960-6bf45fa64465\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"
Jul 09 03:11:01.022503 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:01.022480    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm9mf\" (UniqueName: \"kubernetes.io/projected/e3891296-65af-4372-b960-6bf45fa64465-kube-api-access-cm9mf\") pod \"maas-api-e2e-mt-28444007-7796df9dfd-b6zmq\" (UID: \"e3891296-65af-4372-b960-6bf45fa64465\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"
Jul 09 03:11:01.514788 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:01.514756    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3891296-65af-4372-b960-6bf45fa64465-maas-api-tls\") pod \"maas-api-e2e-mt-28444007-7796df9dfd-b6zmq\" (UID: \"e3891296-65af-4372-b960-6bf45fa64465\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"
Jul 09 03:11:01.517127 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:01.517099    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3891296-65af-4372-b960-6bf45fa64465-maas-api-tls\") pod \"maas-api-e2e-mt-28444007-7796df9dfd-b6zmq\" (UID: \"e3891296-65af-4372-b960-6bf45fa64465\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"
Jul 09 03:11:01.717442 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:01.717408    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"
Jul 09 03:11:01.853515 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:01.853483    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"]
Jul 09 03:11:01.854732 ip-10-0-129-236 kubenswrapper[2578]: W0709 03:11:01.854705    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3891296_65af_4372_b960_6bf45fa64465.slice/crio-b27cea9197924ee8c11e83e58a82c1a37ad5837ea24b8260d63436923cdd61d0 WatchSource:0}: Error finding container b27cea9197924ee8c11e83e58a82c1a37ad5837ea24b8260d63436923cdd61d0: Status 404 returned error can't find the container with id b27cea9197924ee8c11e83e58a82c1a37ad5837ea24b8260d63436923cdd61d0
Jul 09 03:11:01.856212 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:01.856191    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 03:11:02.600921 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:02.600884    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq" event={"ID":"e3891296-65af-4372-b960-6bf45fa64465","Type":"ContainerStarted","Data":"2cb157e39c72bebe0c2687571a62cc6d444a97645a34285054a9a4f36ffa6dcc"}
Jul 09 03:11:02.601109 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:02.600939    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq" event={"ID":"e3891296-65af-4372-b960-6bf45fa64465","Type":"ContainerStarted","Data":"b27cea9197924ee8c11e83e58a82c1a37ad5837ea24b8260d63436923cdd61d0"}
Jul 09 03:11:02.601109 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:02.601079    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"
Jul 09 03:11:02.617679 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:02.617630    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq" podStartSLOduration=2.096198733 podStartE2EDuration="2.617615527s" totalImagesPullingTime="521.416794ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:11:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:11:02.615499065 +0000 UTC m=+2177.009302481" watchObservedRunningTime="2026-07-09 03:11:02.617615527 +0000 UTC m=+2177.011418965"
Jul 09 03:11:06.981448 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:06.981417    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"]
Jul 09 03:11:06.985014 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:06.984993    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:06.987378 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:06.987352    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-28444007\"/\"default-dockercfg-fhxd9\""
Jul 09 03:11:06.987378 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:06.987366    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-28444007\"/\"e2e-lifecycle-model-28444007-kserve-self-signed-certs\""
Jul 09 03:11:06.987544 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:06.987404    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-28444007\"/\"kube-root-ca.crt\""
Jul 09 03:11:06.987544 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:06.987418    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-28444007\"/\"openshift-service-ca.crt\""
Jul 09 03:11:06.994193 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:06.994173    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"]
Jul 09 03:11:07.064933 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.064895    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-model-cache\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.065068 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.064945    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-home\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.065068 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.064969    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-tls-certs\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.065193 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.065070    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-dshm\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.065193 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.065115    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-tmp-dir\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.065193 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.065181    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvpvt\" (UniqueName: \"kubernetes.io/projected/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-kube-api-access-nvpvt\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.165736 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.165701    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-tmp-dir\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.165908 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.165886    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvpvt\" (UniqueName: \"kubernetes.io/projected/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-kube-api-access-nvpvt\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.165993 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.165946    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-model-cache\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.166062 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.165989    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-home\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.166062 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.166015    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-tls-certs\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.166186 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.166167    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-dshm\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.166364 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.166256    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-model-cache\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.166462 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.166406    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-home\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.166462 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.166418    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-tmp-dir\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.168305 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.168272    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-dshm\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.168694 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.168674    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-tls-certs\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.173077 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.173056    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvpvt\" (UniqueName: \"kubernetes.io/projected/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-kube-api-access-nvpvt\") pod \"e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") " pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.297277 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.297216    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:07.428265 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.428236    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"]
Jul 09 03:11:07.430848 ip-10-0-129-236 kubenswrapper[2578]: W0709 03:11:07.430820    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6d6bdf7_4db4_49cb_b22f_737f1ee67cb3.slice/crio-1c282124378548b5fc904b621cf06ced94d298e0e75785d335322ea5b6c72c7b WatchSource:0}: Error finding container 1c282124378548b5fc904b621cf06ced94d298e0e75785d335322ea5b6c72c7b: Status 404 returned error can't find the container with id 1c282124378548b5fc904b621cf06ced94d298e0e75785d335322ea5b6c72c7b
Jul 09 03:11:07.630157 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.630125    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq" event={"ID":"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3","Type":"ContainerStarted","Data":"5b64dd9417cf3c354c5297f13d94dcb11d1c814c6394d1311ccda9720f38d6d0"}
Jul 09 03:11:07.630342 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.630165    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq" event={"ID":"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3","Type":"ContainerStarted","Data":"1c282124378548b5fc904b621cf06ced94d298e0e75785d335322ea5b6c72c7b"}
Jul 09 03:11:07.645489 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:07.645421    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq" podStartSLOduration=1.645400966 podStartE2EDuration="1.645400966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:11:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:11:07.644899328 +0000 UTC m=+2182.038702742" watchObservedRunningTime="2026-07-09 03:11:07.645400966 +0000 UTC m=+2182.039204380"
Jul 09 03:11:08.611176 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:08.611143    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"
Jul 09 03:11:17.297360 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:17.297310    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:17.297360 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:17.297358    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:17.311353 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:17.311330    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:17.689140 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:17.689109    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:11:47.535767 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:47.535728    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"]
Jul 09 03:11:47.536341 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:47.535983    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq" podUID="e3891296-65af-4372-b960-6bf45fa64465" containerName="maas-api" containerID="cri-o://2cb157e39c72bebe0c2687571a62cc6d444a97645a34285054a9a4f36ffa6dcc" gracePeriod=30
Jul 09 03:11:47.782678 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:47.782651    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"
Jul 09 03:11:47.827036 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:47.826959    2578 generic.go:356] "Generic (PLEG): container finished" podID="e3891296-65af-4372-b960-6bf45fa64465" containerID="2cb157e39c72bebe0c2687571a62cc6d444a97645a34285054a9a4f36ffa6dcc" exitCode=0
Jul 09 03:11:47.827036 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:47.827002    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"
Jul 09 03:11:47.827036 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:47.827006    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq" event={"ID":"e3891296-65af-4372-b960-6bf45fa64465","Type":"ContainerDied","Data":"2cb157e39c72bebe0c2687571a62cc6d444a97645a34285054a9a4f36ffa6dcc"}
Jul 09 03:11:47.827290 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:47.827082    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq" event={"ID":"e3891296-65af-4372-b960-6bf45fa64465","Type":"ContainerDied","Data":"b27cea9197924ee8c11e83e58a82c1a37ad5837ea24b8260d63436923cdd61d0"}
Jul 09 03:11:47.827290 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:47.827105    2578 scope.go:117] "RemoveContainer" containerID="2cb157e39c72bebe0c2687571a62cc6d444a97645a34285054a9a4f36ffa6dcc"
Jul 09 03:11:47.835278 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:47.835258    2578 scope.go:117] "RemoveContainer" containerID="2cb157e39c72bebe0c2687571a62cc6d444a97645a34285054a9a4f36ffa6dcc"
Jul 09 03:11:47.835576 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:11:47.835557    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cb157e39c72bebe0c2687571a62cc6d444a97645a34285054a9a4f36ffa6dcc\": container with ID starting with 2cb157e39c72bebe0c2687571a62cc6d444a97645a34285054a9a4f36ffa6dcc not found: ID does not exist" containerID="2cb157e39c72bebe0c2687571a62cc6d444a97645a34285054a9a4f36ffa6dcc"
Jul 09 03:11:47.835656 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:47.835588    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cb157e39c72bebe0c2687571a62cc6d444a97645a34285054a9a4f36ffa6dcc"} err="failed to get container status \"2cb157e39c72bebe0c2687571a62cc6d444a97645a34285054a9a4f36ffa6dcc\": rpc error: code = NotFound desc = could not find container \"2cb157e39c72bebe0c2687571a62cc6d444a97645a34285054a9a4f36ffa6dcc\": container with ID starting with 2cb157e39c72bebe0c2687571a62cc6d444a97645a34285054a9a4f36ffa6dcc not found: ID does not exist"
Jul 09 03:11:47.912633 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:47.912609    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cm9mf\" (UniqueName: \"kubernetes.io/projected/e3891296-65af-4372-b960-6bf45fa64465-kube-api-access-cm9mf\") pod \"e3891296-65af-4372-b960-6bf45fa64465\" (UID: \"e3891296-65af-4372-b960-6bf45fa64465\") "
Jul 09 03:11:47.912727 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:47.912655    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3891296-65af-4372-b960-6bf45fa64465-openshift-service-ca\") pod \"e3891296-65af-4372-b960-6bf45fa64465\" (UID: \"e3891296-65af-4372-b960-6bf45fa64465\") "
Jul 09 03:11:47.912773 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:47.912741    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3891296-65af-4372-b960-6bf45fa64465-maas-api-tls\") pod \"e3891296-65af-4372-b960-6bf45fa64465\" (UID: \"e3891296-65af-4372-b960-6bf45fa64465\") "
Jul 09 03:11:47.913053 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:47.913028    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3891296-65af-4372-b960-6bf45fa64465-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e3891296-65af-4372-b960-6bf45fa64465" (UID: "e3891296-65af-4372-b960-6bf45fa64465"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 03:11:47.914516 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:47.914493    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3891296-65af-4372-b960-6bf45fa64465-kube-api-access-cm9mf" (OuterVolumeSpecName: "kube-api-access-cm9mf") pod "e3891296-65af-4372-b960-6bf45fa64465" (UID: "e3891296-65af-4372-b960-6bf45fa64465"). InnerVolumeSpecName "kube-api-access-cm9mf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:11:47.914707 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:47.914689    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3891296-65af-4372-b960-6bf45fa64465-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e3891296-65af-4372-b960-6bf45fa64465" (UID: "e3891296-65af-4372-b960-6bf45fa64465"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 03:11:48.013369 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:48.013345    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3891296-65af-4372-b960-6bf45fa64465-maas-api-tls\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:11:48.013369 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:48.013369    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cm9mf\" (UniqueName: \"kubernetes.io/projected/e3891296-65af-4372-b960-6bf45fa64465-kube-api-access-cm9mf\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:11:48.013505 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:48.013384    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3891296-65af-4372-b960-6bf45fa64465-openshift-service-ca\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:11:48.141249 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:48.141221    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"]
Jul 09 03:11:48.144882 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:48.144861    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-28444007-7796df9dfd-b6zmq"]
Jul 09 03:11:50.117849 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:50.117816    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3891296-65af-4372-b960-6bf45fa64465" path="/var/lib/kubelet/pods/e3891296-65af-4372-b960-6bf45fa64465/volumes"
Jul 09 03:11:53.521058 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:11:53.521023    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"]
Jul 09 03:12:08.541469 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.541367    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq" podUID="e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3" containerName="main" containerID="cri-o://5b64dd9417cf3c354c5297f13d94dcb11d1c814c6394d1311ccda9720f38d6d0" gracePeriod=45
Jul 09 03:12:08.788984 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.788962    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:12:08.888990 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.888963    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-tls-certs\") pod \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") "
Jul 09 03:12:08.889173 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.889035    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-dshm\") pod \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") "
Jul 09 03:12:08.889173 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.889053    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-home\") pod \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") "
Jul 09 03:12:08.889173 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.889089    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvpvt\" (UniqueName: \"kubernetes.io/projected/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-kube-api-access-nvpvt\") pod \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") "
Jul 09 03:12:08.889173 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.889119    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-model-cache\") pod \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") "
Jul 09 03:12:08.889430 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.889178    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-tmp-dir\") pod \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\" (UID: \"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3\") "
Jul 09 03:12:08.889430 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.889370    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-home" (OuterVolumeSpecName: "home") pod "e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3" (UID: "e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:12:08.889537 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.889429    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-model-cache" (OuterVolumeSpecName: "model-cache") pod "e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3" (UID: "e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:12:08.889537 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.889496    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3" (UID: "e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:12:08.889537 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.889512    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-home\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:12:08.891260 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.891240    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-dshm" (OuterVolumeSpecName: "dshm") pod "e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3" (UID: "e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:12:08.891347 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.891302    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-kube-api-access-nvpvt" (OuterVolumeSpecName: "kube-api-access-nvpvt") pod "e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3" (UID: "e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3"). InnerVolumeSpecName "kube-api-access-nvpvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:12:08.891453 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.891435    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3" (UID: "e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 03:12:08.935556 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.935527    2578 generic.go:356] "Generic (PLEG): container finished" podID="e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3" containerID="5b64dd9417cf3c354c5297f13d94dcb11d1c814c6394d1311ccda9720f38d6d0" exitCode=0
Jul 09 03:12:08.935677 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.935575    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"
Jul 09 03:12:08.935677 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.935611    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq" event={"ID":"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3","Type":"ContainerDied","Data":"5b64dd9417cf3c354c5297f13d94dcb11d1c814c6394d1311ccda9720f38d6d0"}
Jul 09 03:12:08.935677 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.935651    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq" event={"ID":"e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3","Type":"ContainerDied","Data":"1c282124378548b5fc904b621cf06ced94d298e0e75785d335322ea5b6c72c7b"}
Jul 09 03:12:08.935677 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.935668    2578 scope.go:117] "RemoveContainer" containerID="5b64dd9417cf3c354c5297f13d94dcb11d1c814c6394d1311ccda9720f38d6d0"
Jul 09 03:12:08.944857 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.944836    2578 scope.go:117] "RemoveContainer" containerID="5b64dd9417cf3c354c5297f13d94dcb11d1c814c6394d1311ccda9720f38d6d0"
Jul 09 03:12:08.945088 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:12:08.945070    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b64dd9417cf3c354c5297f13d94dcb11d1c814c6394d1311ccda9720f38d6d0\": container with ID starting with 5b64dd9417cf3c354c5297f13d94dcb11d1c814c6394d1311ccda9720f38d6d0 not found: ID does not exist" containerID="5b64dd9417cf3c354c5297f13d94dcb11d1c814c6394d1311ccda9720f38d6d0"
Jul 09 03:12:08.945138 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.945095    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b64dd9417cf3c354c5297f13d94dcb11d1c814c6394d1311ccda9720f38d6d0"} err="failed to get container status \"5b64dd9417cf3c354c5297f13d94dcb11d1c814c6394d1311ccda9720f38d6d0\": rpc error: code = NotFound desc = could not find container \"5b64dd9417cf3c354c5297f13d94dcb11d1c814c6394d1311ccda9720f38d6d0\": container with ID starting with 5b64dd9417cf3c354c5297f13d94dcb11d1c814c6394d1311ccda9720f38d6d0 not found: ID does not exist"
Jul 09 03:12:08.955653 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.955623    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"]
Jul 09 03:12:08.959076 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.959056    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-28444007/e2e-lifecycle-model-28444007-kserve-586b65d5ff-jcvdq"]
Jul 09 03:12:08.990534 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.990506    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvpvt\" (UniqueName: \"kubernetes.io/projected/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-kube-api-access-nvpvt\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:12:08.990534 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.990527    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-model-cache\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:12:08.990534 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.990538    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-tmp-dir\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:12:08.990677 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.990545    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-tls-certs\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:12:08.990677 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:08.990553    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3-dshm\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:12:10.117949 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:12:10.117906    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3" path="/var/lib/kubelet/pods/e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3/volumes"
Jul 09 03:13:33.746780 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.746741    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"]
Jul 09 03:13:33.747392 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.747371    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3891296-65af-4372-b960-6bf45fa64465" containerName="maas-api"
Jul 09 03:13:33.747472 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.747396    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3891296-65af-4372-b960-6bf45fa64465" containerName="maas-api"
Jul 09 03:13:33.747472 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.747416    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3" containerName="main"
Jul 09 03:13:33.747472 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.747425    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3" containerName="main"
Jul 09 03:13:33.747629 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.747540    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3891296-65af-4372-b960-6bf45fa64465" containerName="maas-api"
Jul 09 03:13:33.747629 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.747554    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6d6bdf7-4db4-49cb-b22f-737f1ee67cb3" containerName="main"
Jul 09 03:13:33.751300 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.751278    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"
Jul 09 03:13:33.753545 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.753525    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-cf3835-serving-cert\""
Jul 09 03:13:33.761002 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.760979    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"]
Jul 09 03:13:33.780211 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.780179    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4e0571-9053-449f-bddc-3d6ba66cea95-openshift-service-ca\") pod \"maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg\" (UID: \"3b4e0571-9053-449f-bddc-3d6ba66cea95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"
Jul 09 03:13:33.780373 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.780226    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8whfq\" (UniqueName: \"kubernetes.io/projected/3b4e0571-9053-449f-bddc-3d6ba66cea95-kube-api-access-8whfq\") pod \"maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg\" (UID: \"3b4e0571-9053-449f-bddc-3d6ba66cea95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"
Jul 09 03:13:33.780373 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.780260    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4e0571-9053-449f-bddc-3d6ba66cea95-maas-api-tls\") pod \"maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg\" (UID: \"3b4e0571-9053-449f-bddc-3d6ba66cea95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"
Jul 09 03:13:33.880644 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.880618    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4e0571-9053-449f-bddc-3d6ba66cea95-openshift-service-ca\") pod \"maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg\" (UID: \"3b4e0571-9053-449f-bddc-3d6ba66cea95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"
Jul 09 03:13:33.880752 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.880670    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8whfq\" (UniqueName: \"kubernetes.io/projected/3b4e0571-9053-449f-bddc-3d6ba66cea95-kube-api-access-8whfq\") pod \"maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg\" (UID: \"3b4e0571-9053-449f-bddc-3d6ba66cea95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"
Jul 09 03:13:33.880752 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.880703    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4e0571-9053-449f-bddc-3d6ba66cea95-maas-api-tls\") pod \"maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg\" (UID: \"3b4e0571-9053-449f-bddc-3d6ba66cea95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"
Jul 09 03:13:33.880824 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:13:33.880807    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-serving-cert: secret "maas-api-e2e-inf-a-cf3835-serving-cert" not found
Jul 09 03:13:33.880875 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:13:33.880870    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b4e0571-9053-449f-bddc-3d6ba66cea95-maas-api-tls podName:3b4e0571-9053-449f-bddc-3d6ba66cea95 nodeName:}" failed. No retries permitted until 2026-07-09 03:13:34.380854703 +0000 UTC m=+2328.774658093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3b4e0571-9053-449f-bddc-3d6ba66cea95-maas-api-tls") pod "maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg" (UID: "3b4e0571-9053-449f-bddc-3d6ba66cea95") : secret "maas-api-e2e-inf-a-cf3835-serving-cert" not found
Jul 09 03:13:33.881362 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.881343    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4e0571-9053-449f-bddc-3d6ba66cea95-openshift-service-ca\") pod \"maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg\" (UID: \"3b4e0571-9053-449f-bddc-3d6ba66cea95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"
Jul 09 03:13:33.888814 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:33.888795    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8whfq\" (UniqueName: \"kubernetes.io/projected/3b4e0571-9053-449f-bddc-3d6ba66cea95-kube-api-access-8whfq\") pod \"maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg\" (UID: \"3b4e0571-9053-449f-bddc-3d6ba66cea95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"
Jul 09 03:13:34.385821 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:34.385789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4e0571-9053-449f-bddc-3d6ba66cea95-maas-api-tls\") pod \"maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg\" (UID: \"3b4e0571-9053-449f-bddc-3d6ba66cea95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"
Jul 09 03:13:34.388190 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:34.388166    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4e0571-9053-449f-bddc-3d6ba66cea95-maas-api-tls\") pod \"maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg\" (UID: \"3b4e0571-9053-449f-bddc-3d6ba66cea95\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"
Jul 09 03:13:34.662993 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:34.662926    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"
Jul 09 03:13:34.791156 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:34.791131    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"]
Jul 09 03:13:34.793659 ip-10-0-129-236 kubenswrapper[2578]: W0709 03:13:34.793627    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b4e0571_9053_449f_bddc_3d6ba66cea95.slice/crio-afbc7a21436ffa4098710656049c3fa3ab5be1f99811874c8d7725110c82280d WatchSource:0}: Error finding container afbc7a21436ffa4098710656049c3fa3ab5be1f99811874c8d7725110c82280d: Status 404 returned error can't find the container with id afbc7a21436ffa4098710656049c3fa3ab5be1f99811874c8d7725110c82280d
Jul 09 03:13:35.362839 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:35.362806    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg" event={"ID":"3b4e0571-9053-449f-bddc-3d6ba66cea95","Type":"ContainerStarted","Data":"3e7335d4efbe1a13d705ff5f704dba67f5a5af5b91d1af9c10799a151bf299e8"}
Jul 09 03:13:35.362943 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:35.362902    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg" event={"ID":"3b4e0571-9053-449f-bddc-3d6ba66cea95","Type":"ContainerStarted","Data":"afbc7a21436ffa4098710656049c3fa3ab5be1f99811874c8d7725110c82280d"}
Jul 09 03:13:35.362943 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:35.362935    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"
Jul 09 03:13:35.380186 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:35.380144    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg" podStartSLOduration=1.882668505 podStartE2EDuration="2.380128349s" totalImagesPullingTime="497.459844ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:13:35.379073085 +0000 UTC m=+2329.772876501" watchObservedRunningTime="2026-07-09 03:13:35.380128349 +0000 UTC m=+2329.773931762"
Jul 09 03:13:38.450102 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.449894    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"]
Jul 09 03:13:38.456276 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.456251    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.458766 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.458741    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-defa37-openshift-default-dockercfg-mqkx4\""
Jul 09 03:13:38.464847 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.464822    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"]
Jul 09 03:13:38.520044 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.520019    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-workload-certs\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.520158 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.520050    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-data\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.520210 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.520156    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istiod-ca-cert\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.520210 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.520175    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-token\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.520361 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.520225    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-envoy\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.520361 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.520250    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkqhc\" (UniqueName: \"kubernetes.io/projected/29ea98d1-701b-4fb1-9bda-7c1979212fd5-kube-api-access-vkqhc\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.520361 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.520273    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-credential-socket\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.520361 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.520302    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-workload-socket\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.520361 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.520331    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-podinfo\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.620858 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.620834    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istiod-ca-cert\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.620997 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.620873    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-token\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.620997 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.620913    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-envoy\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.620997 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.620936    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkqhc\" (UniqueName: \"kubernetes.io/projected/29ea98d1-701b-4fb1-9bda-7c1979212fd5-kube-api-access-vkqhc\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.620997 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.620966    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-credential-socket\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.621211 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.621045    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-workload-socket\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.621211 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.621089    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-podinfo\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.621211 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.621136    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-workload-certs\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.621211 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.621167    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-data\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.621502 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.621479    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-workload-socket\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.621569 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.621544    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-data\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.621661 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.621638    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istiod-ca-cert\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.621717 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.621639    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-credential-socket\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.621773 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.621735    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-workload-certs\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.623434 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.623341    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-envoy\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.624511 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.624485    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-podinfo\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.627893 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.627865    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-token\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.628081 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.628064    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkqhc\" (UniqueName: \"kubernetes.io/projected/29ea98d1-701b-4fb1-9bda-7c1979212fd5-kube-api-access-vkqhc\") pod \"e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") " pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.768381 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.768311    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:38.902418 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:38.902388    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"]
Jul 09 03:13:38.903663 ip-10-0-129-236 kubenswrapper[2578]: W0709 03:13:38.903639    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ea98d1_701b_4fb1_9bda_7c1979212fd5.slice/crio-623d902eab458f81cee6ebff04b19141e826a70a4c81dfdde6a42a92f1613261 WatchSource:0}: Error finding container 623d902eab458f81cee6ebff04b19141e826a70a4c81dfdde6a42a92f1613261: Status 404 returned error can't find the container with id 623d902eab458f81cee6ebff04b19141e826a70a4c81dfdde6a42a92f1613261
Jul 09 03:13:39.383243 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:39.383209    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj" event={"ID":"29ea98d1-701b-4fb1-9bda-7c1979212fd5","Type":"ContainerStarted","Data":"623d902eab458f81cee6ebff04b19141e826a70a4c81dfdde6a42a92f1613261"}
Jul 09 03:13:41.374157 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:41.374122    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"
Jul 09 03:13:42.137893 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:42.137852    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 03:13:42.138036 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:42.137931    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 03:13:42.138036 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:42.137962    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 03:13:42.404030 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:42.403993    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj" event={"ID":"29ea98d1-701b-4fb1-9bda-7c1979212fd5","Type":"ContainerStarted","Data":"e6eb7fa578c29938a250863697459b8daa037398576ab3181398e9b0a86efb49"}
Jul 09 03:13:42.421787 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:42.421734    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj" podStartSLOduration=1.189918628 podStartE2EDuration="4.421716787s" totalImagesPullingTime="3.231798159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:13:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:13:42.42045115 +0000 UTC m=+2336.814254563" watchObservedRunningTime="2026-07-09 03:13:42.421716787 +0000 UTC m=+2336.815520201"
Jul 09 03:13:42.769370 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:42.769275    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:43.774133 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:43.774105    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:44.414271 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:44.414240    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:44.415035 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:44.415018    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:44.520055 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:44.520024    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"]
Jul 09 03:13:44.523775 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:44.523754    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"
Jul 09 03:13:44.525796 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:44.525776    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-defa37-serving-cert\""
Jul 09 03:13:44.533044 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:44.533018    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"]
Jul 09 03:13:44.571711 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:44.571684    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d559e056-c5fb-464c-9dcd-f9c18f3741d0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg\" (UID: \"d559e056-c5fb-464c-9dcd-f9c18f3741d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"
Jul 09 03:13:44.571858 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:44.571749    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d559e056-c5fb-464c-9dcd-f9c18f3741d0-maas-api-tls\") pod \"maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg\" (UID: \"d559e056-c5fb-464c-9dcd-f9c18f3741d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"
Jul 09 03:13:44.571858 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:44.571788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkj8d\" (UniqueName: \"kubernetes.io/projected/d559e056-c5fb-464c-9dcd-f9c18f3741d0-kube-api-access-tkj8d\") pod \"maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg\" (UID: \"d559e056-c5fb-464c-9dcd-f9c18f3741d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"
Jul 09 03:13:44.673180 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:44.673097    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d559e056-c5fb-464c-9dcd-f9c18f3741d0-maas-api-tls\") pod \"maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg\" (UID: \"d559e056-c5fb-464c-9dcd-f9c18f3741d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"
Jul 09 03:13:44.673180 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:44.673132    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkj8d\" (UniqueName: \"kubernetes.io/projected/d559e056-c5fb-464c-9dcd-f9c18f3741d0-kube-api-access-tkj8d\") pod \"maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg\" (UID: \"d559e056-c5fb-464c-9dcd-f9c18f3741d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"
Jul 09 03:13:44.673180 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:44.673180    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d559e056-c5fb-464c-9dcd-f9c18f3741d0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg\" (UID: \"d559e056-c5fb-464c-9dcd-f9c18f3741d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"
Jul 09 03:13:44.673548 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:13:44.673248    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-serving-cert: secret "maas-api-e2e-inf-b-defa37-serving-cert" not found
Jul 09 03:13:44.673548 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:13:44.673364    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d559e056-c5fb-464c-9dcd-f9c18f3741d0-maas-api-tls podName:d559e056-c5fb-464c-9dcd-f9c18f3741d0 nodeName:}" failed. No retries permitted until 2026-07-09 03:13:45.17334289 +0000 UTC m=+2339.567146298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d559e056-c5fb-464c-9dcd-f9c18f3741d0-maas-api-tls") pod "maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg" (UID: "d559e056-c5fb-464c-9dcd-f9c18f3741d0") : secret "maas-api-e2e-inf-b-defa37-serving-cert" not found
Jul 09 03:13:44.673782 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:44.673765    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d559e056-c5fb-464c-9dcd-f9c18f3741d0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg\" (UID: \"d559e056-c5fb-464c-9dcd-f9c18f3741d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"
Jul 09 03:13:44.681481 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:44.681459    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkj8d\" (UniqueName: \"kubernetes.io/projected/d559e056-c5fb-464c-9dcd-f9c18f3741d0-kube-api-access-tkj8d\") pod \"maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg\" (UID: \"d559e056-c5fb-464c-9dcd-f9c18f3741d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"
Jul 09 03:13:45.179377 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:45.179342    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d559e056-c5fb-464c-9dcd-f9c18f3741d0-maas-api-tls\") pod \"maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg\" (UID: \"d559e056-c5fb-464c-9dcd-f9c18f3741d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"
Jul 09 03:13:45.181747 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:45.181726    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d559e056-c5fb-464c-9dcd-f9c18f3741d0-maas-api-tls\") pod \"maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg\" (UID: \"d559e056-c5fb-464c-9dcd-f9c18f3741d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"
Jul 09 03:13:45.435214 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:45.435130    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"
Jul 09 03:13:45.568330 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:45.568288    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"]
Jul 09 03:13:45.569941 ip-10-0-129-236 kubenswrapper[2578]: W0709 03:13:45.569903    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd559e056_c5fb_464c_9dcd_f9c18f3741d0.slice/crio-3f1d79e7a7666e387b944e37056e697b15ae295b7e0dc5a1803fc0b2e7097f15 WatchSource:0}: Error finding container 3f1d79e7a7666e387b944e37056e697b15ae295b7e0dc5a1803fc0b2e7097f15: Status 404 returned error can't find the container with id 3f1d79e7a7666e387b944e37056e697b15ae295b7e0dc5a1803fc0b2e7097f15
Jul 09 03:13:46.424606 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:46.424571    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg" event={"ID":"d559e056-c5fb-464c-9dcd-f9c18f3741d0","Type":"ContainerStarted","Data":"e0105a964da7871017e997cff46f56edcaffc794771b6de8520ee0274de663f5"}
Jul 09 03:13:46.425005 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:46.424755    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"
Jul 09 03:13:46.425005 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:46.424789    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg" event={"ID":"d559e056-c5fb-464c-9dcd-f9c18f3741d0","Type":"ContainerStarted","Data":"3f1d79e7a7666e387b944e37056e697b15ae295b7e0dc5a1803fc0b2e7097f15"}
Jul 09 03:13:46.445806 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:46.445759    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg" podStartSLOduration=2.060589789 podStartE2EDuration="2.44574334s" totalImagesPullingTime="385.153551ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:13:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:13:46.443743313 +0000 UTC m=+2340.837546728" watchObservedRunningTime="2026-07-09 03:13:46.44574334 +0000 UTC m=+2340.839546754"
Jul 09 03:13:52.440748 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:52.440707    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"
Jul 09 03:13:56.490688 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:56.490653    2578 generic.go:356] "Generic (PLEG): container finished" podID="29ea98d1-701b-4fb1-9bda-7c1979212fd5" containerID="e6eb7fa578c29938a250863697459b8daa037398576ab3181398e9b0a86efb49" exitCode=0
Jul 09 03:13:56.491104 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:56.490699    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj" event={"ID":"29ea98d1-701b-4fb1-9bda-7c1979212fd5","Type":"ContainerDied","Data":"e6eb7fa578c29938a250863697459b8daa037398576ab3181398e9b0a86efb49"}
Jul 09 03:13:56.491104 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:56.491046    2578 scope.go:117] "RemoveContainer" containerID="e6eb7fa578c29938a250863697459b8daa037398576ab3181398e9b0a86efb49"
Jul 09 03:13:56.492050 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:56.492017    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 03:13:56.492115 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:56.492101    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 03:13:56.492152 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:56.492135    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 03:13:57.498371 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:57.498338    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj" event={"ID":"29ea98d1-701b-4fb1-9bda-7c1979212fd5","Type":"ContainerStarted","Data":"0e19dcc32f8e3192a9c308820c62e845281d4855802981afada7a4e3ec35de75"}
Jul 09 03:13:57.768834 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:57.768736    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:57.770016 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:57.769989    2578 patch_prober.go:28] interesting pod/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 09 03:13:57.770136 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:57.770040    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj" podUID="29ea98d1-701b-4fb1-9bda-7c1979212fd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 09 03:13:58.769769 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:58.769729    2578 patch_prober.go:28] interesting pod/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 09 03:13:58.770161 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:58.769806    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj" podUID="29ea98d1-701b-4fb1-9bda-7c1979212fd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 09 03:13:59.414709 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:59.414672    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:13:59.769519 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:59.769426    2578 patch_prober.go:28] interesting pod/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 09 03:13:59.769519 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:13:59.769507    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj" podUID="29ea98d1-701b-4fb1-9bda-7c1979212fd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 09 03:14:00.768901 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:14:00.768864    2578 patch_prober.go:28] interesting pod/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 09 03:14:00.769293 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:14:00.768925    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj" podUID="29ea98d1-701b-4fb1-9bda-7c1979212fd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 09 03:14:01.772262 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:14:01.772232    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:14:01.773283 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:14:01.773263    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:15:00.140136 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.140099    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf"]
Jul 09 03:15:00.143854 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.143832    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r"]
Jul 09 03:15:00.143991 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.143941    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf"
Jul 09 03:15:00.147336 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.147301    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p"]
Jul 09 03:15:00.147544 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.147523    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r"
Jul 09 03:15:00.151072 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.151048    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf"]
Jul 09 03:15:00.151179 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.151144    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p"
Jul 09 03:15:00.153146 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.153122    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r"]
Jul 09 03:15:00.155857 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.155833    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p"]
Jul 09 03:15:00.333542 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.333508    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6kzs\" (UniqueName: \"kubernetes.io/projected/96b46041-c5b7-495d-ba2a-d68276259891-kube-api-access-v6kzs\") pod \"maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r\" (UID: \"96b46041-c5b7-495d-ba2a-d68276259891\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r"
Jul 09 03:15:00.333700 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.333583    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7q2r\" (UniqueName: \"kubernetes.io/projected/b3cb093f-1c42-4526-9b82-d277f54f1815-kube-api-access-c7q2r\") pod \"maas-api-key-cleanup-29726115-n6z8p\" (UID: \"b3cb093f-1c42-4526-9b82-d277f54f1815\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p"
Jul 09 03:15:00.333700 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.333664    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85tfp\" (UniqueName: \"kubernetes.io/projected/9c863ceb-0117-4e7b-ba06-27253577f37c-kube-api-access-85tfp\") pod \"maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf\" (UID: \"9c863ceb-0117-4e7b-ba06-27253577f37c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf"
Jul 09 03:15:00.434579 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.434508    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7q2r\" (UniqueName: \"kubernetes.io/projected/b3cb093f-1c42-4526-9b82-d277f54f1815-kube-api-access-c7q2r\") pod \"maas-api-key-cleanup-29726115-n6z8p\" (UID: \"b3cb093f-1c42-4526-9b82-d277f54f1815\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p"
Jul 09 03:15:00.434579 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.434573    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85tfp\" (UniqueName: \"kubernetes.io/projected/9c863ceb-0117-4e7b-ba06-27253577f37c-kube-api-access-85tfp\") pod \"maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf\" (UID: \"9c863ceb-0117-4e7b-ba06-27253577f37c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf"
Jul 09 03:15:00.434732 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.434606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6kzs\" (UniqueName: \"kubernetes.io/projected/96b46041-c5b7-495d-ba2a-d68276259891-kube-api-access-v6kzs\") pod \"maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r\" (UID: \"96b46041-c5b7-495d-ba2a-d68276259891\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r"
Jul 09 03:15:00.442389 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.442366    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7q2r\" (UniqueName: \"kubernetes.io/projected/b3cb093f-1c42-4526-9b82-d277f54f1815-kube-api-access-c7q2r\") pod \"maas-api-key-cleanup-29726115-n6z8p\" (UID: \"b3cb093f-1c42-4526-9b82-d277f54f1815\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p"
Jul 09 03:15:00.442503 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.442374    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6kzs\" (UniqueName: \"kubernetes.io/projected/96b46041-c5b7-495d-ba2a-d68276259891-kube-api-access-v6kzs\") pod \"maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r\" (UID: \"96b46041-c5b7-495d-ba2a-d68276259891\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r"
Jul 09 03:15:00.442503 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.442410    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85tfp\" (UniqueName: \"kubernetes.io/projected/9c863ceb-0117-4e7b-ba06-27253577f37c-kube-api-access-85tfp\") pod \"maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf\" (UID: \"9c863ceb-0117-4e7b-ba06-27253577f37c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf"
Jul 09 03:15:00.459094 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.459075    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf"
Jul 09 03:15:00.467142 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.467127    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r"
Jul 09 03:15:00.473019 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.472972    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p"
Jul 09 03:15:00.613174 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.613109    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf"]
Jul 09 03:15:00.615780 ip-10-0-129-236 kubenswrapper[2578]: W0709 03:15:00.615475    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c863ceb_0117_4e7b_ba06_27253577f37c.slice/crio-10bca3ef53ecc2f45ba464947f28106e2e2f7c5d1b0fbd0fe3c0b0683630dc72 WatchSource:0}: Error finding container 10bca3ef53ecc2f45ba464947f28106e2e2f7c5d1b0fbd0fe3c0b0683630dc72: Status 404 returned error can't find the container with id 10bca3ef53ecc2f45ba464947f28106e2e2f7c5d1b0fbd0fe3c0b0683630dc72
Jul 09 03:15:00.824953 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.824888    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf" event={"ID":"9c863ceb-0117-4e7b-ba06-27253577f37c","Type":"ContainerStarted","Data":"10bca3ef53ecc2f45ba464947f28106e2e2f7c5d1b0fbd0fe3c0b0683630dc72"}
Jul 09 03:15:00.847807 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.847774    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r"]
Jul 09 03:15:00.849671 ip-10-0-129-236 kubenswrapper[2578]: W0709 03:15:00.849631    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b46041_c5b7_495d_ba2a_d68276259891.slice/crio-7fa8bb30cef4b4eb9eba3f229e79b885efa5979ae7cb1a4633bf093839293f9c WatchSource:0}: Error finding container 7fa8bb30cef4b4eb9eba3f229e79b885efa5979ae7cb1a4633bf093839293f9c: Status 404 returned error can't find the container with id 7fa8bb30cef4b4eb9eba3f229e79b885efa5979ae7cb1a4633bf093839293f9c
Jul 09 03:15:00.850288 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:00.850230    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p"]
Jul 09 03:15:00.852364 ip-10-0-129-236 kubenswrapper[2578]: W0709 03:15:00.852299    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3cb093f_1c42_4526_9b82_d277f54f1815.slice/crio-848228d53fa08010131d31961ad03e58a75ff9a44e2797cc3da673282f58c93a WatchSource:0}: Error finding container 848228d53fa08010131d31961ad03e58a75ff9a44e2797cc3da673282f58c93a: Status 404 returned error can't find the container with id 848228d53fa08010131d31961ad03e58a75ff9a44e2797cc3da673282f58c93a
Jul 09 03:15:01.831589 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:01.831548    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf" event={"ID":"9c863ceb-0117-4e7b-ba06-27253577f37c","Type":"ContainerStarted","Data":"ede59db66dc58dafb343299087af8c63681e410d41d8145b73daa3b1404ddb25"}
Jul 09 03:15:01.833220 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:01.833195    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p" event={"ID":"b3cb093f-1c42-4526-9b82-d277f54f1815","Type":"ContainerStarted","Data":"7cf18f7ef701c0fe828c0a69a16a6859b41a8c4f09c3b6e1281c2b4106eee0d7"}
Jul 09 03:15:01.833351 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:01.833241    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p" event={"ID":"b3cb093f-1c42-4526-9b82-d277f54f1815","Type":"ContainerStarted","Data":"848228d53fa08010131d31961ad03e58a75ff9a44e2797cc3da673282f58c93a"}
Jul 09 03:15:01.834629 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:01.834603    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r" event={"ID":"96b46041-c5b7-495d-ba2a-d68276259891","Type":"ContainerStarted","Data":"17d1bbaec0384c939e7207aefa530ee5f7e761efedbe2f16a0df564b319c747e"}
Jul 09 03:15:01.834726 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:01.834651    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r" event={"ID":"96b46041-c5b7-495d-ba2a-d68276259891","Type":"ContainerStarted","Data":"7fa8bb30cef4b4eb9eba3f229e79b885efa5979ae7cb1a4633bf093839293f9c"}
Jul 09 03:15:01.847243 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:01.847204    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf" podStartSLOduration=1.847192444 podStartE2EDuration="1.847192444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:15:01.844894227 +0000 UTC m=+2416.238697640" watchObservedRunningTime="2026-07-09 03:15:01.847192444 +0000 UTC m=+2416.240995857"
Jul 09 03:15:01.858355 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:01.858297    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r" podStartSLOduration=1.858287925 podStartE2EDuration="1.858287925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:15:01.857449575 +0000 UTC m=+2416.251252988" watchObservedRunningTime="2026-07-09 03:15:01.858287925 +0000 UTC m=+2416.252091338"
Jul 09 03:15:01.873700 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:01.873664    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p" podStartSLOduration=1.873652825 podStartE2EDuration="1.873652825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:15:01.871571321 +0000 UTC m=+2416.265374735" watchObservedRunningTime="2026-07-09 03:15:01.873652825 +0000 UTC m=+2416.267456238"
Jul 09 03:15:04.322295 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.322261    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"]
Jul 09 03:15:04.323068 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.323036    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg" podUID="3b4e0571-9053-449f-bddc-3d6ba66cea95" containerName="maas-api" containerID="cri-o://3e7335d4efbe1a13d705ff5f704dba67f5a5af5b91d1af9c10799a151bf299e8" gracePeriod=30
Jul 09 03:15:04.359398 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.359371    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r"]
Jul 09 03:15:04.359573 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.359549    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r" podUID="96b46041-c5b7-495d-ba2a-d68276259891" containerName="cleanup" containerID="cri-o://17d1bbaec0384c939e7207aefa530ee5f7e761efedbe2f16a0df564b319c747e" gracePeriod=30
Jul 09 03:15:04.575181 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.575123    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"
Jul 09 03:15:04.677627 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.677596    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4e0571-9053-449f-bddc-3d6ba66cea95-maas-api-tls\") pod \"3b4e0571-9053-449f-bddc-3d6ba66cea95\" (UID: \"3b4e0571-9053-449f-bddc-3d6ba66cea95\") "
Jul 09 03:15:04.677790 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.677646    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4e0571-9053-449f-bddc-3d6ba66cea95-openshift-service-ca\") pod \"3b4e0571-9053-449f-bddc-3d6ba66cea95\" (UID: \"3b4e0571-9053-449f-bddc-3d6ba66cea95\") "
Jul 09 03:15:04.677790 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.677763    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8whfq\" (UniqueName: \"kubernetes.io/projected/3b4e0571-9053-449f-bddc-3d6ba66cea95-kube-api-access-8whfq\") pod \"3b4e0571-9053-449f-bddc-3d6ba66cea95\" (UID: \"3b4e0571-9053-449f-bddc-3d6ba66cea95\") "
Jul 09 03:15:04.678030 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.678008    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b4e0571-9053-449f-bddc-3d6ba66cea95-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b4e0571-9053-449f-bddc-3d6ba66cea95" (UID: "3b4e0571-9053-449f-bddc-3d6ba66cea95"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 03:15:04.679674 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.679651    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b4e0571-9053-449f-bddc-3d6ba66cea95-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b4e0571-9053-449f-bddc-3d6ba66cea95" (UID: "3b4e0571-9053-449f-bddc-3d6ba66cea95"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 03:15:04.679760 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.679740    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b4e0571-9053-449f-bddc-3d6ba66cea95-kube-api-access-8whfq" (OuterVolumeSpecName: "kube-api-access-8whfq") pod "3b4e0571-9053-449f-bddc-3d6ba66cea95" (UID: "3b4e0571-9053-449f-bddc-3d6ba66cea95"). InnerVolumeSpecName "kube-api-access-8whfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:15:04.778667 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.778635    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b4e0571-9053-449f-bddc-3d6ba66cea95-maas-api-tls\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:15:04.778667 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.778664    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4e0571-9053-449f-bddc-3d6ba66cea95-openshift-service-ca\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:15:04.778848 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.778679    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8whfq\" (UniqueName: \"kubernetes.io/projected/3b4e0571-9053-449f-bddc-3d6ba66cea95-kube-api-access-8whfq\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:15:04.851031 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.850953    2578 generic.go:356] "Generic (PLEG): container finished" podID="3b4e0571-9053-449f-bddc-3d6ba66cea95" containerID="3e7335d4efbe1a13d705ff5f704dba67f5a5af5b91d1af9c10799a151bf299e8" exitCode=0
Jul 09 03:15:04.851031 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.851000    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"
Jul 09 03:15:04.851188 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.851036    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg" event={"ID":"3b4e0571-9053-449f-bddc-3d6ba66cea95","Type":"ContainerDied","Data":"3e7335d4efbe1a13d705ff5f704dba67f5a5af5b91d1af9c10799a151bf299e8"}
Jul 09 03:15:04.851188 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.851090    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg" event={"ID":"3b4e0571-9053-449f-bddc-3d6ba66cea95","Type":"ContainerDied","Data":"afbc7a21436ffa4098710656049c3fa3ab5be1f99811874c8d7725110c82280d"}
Jul 09 03:15:04.851188 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.851113    2578 scope.go:117] "RemoveContainer" containerID="3e7335d4efbe1a13d705ff5f704dba67f5a5af5b91d1af9c10799a151bf299e8"
Jul 09 03:15:04.860179 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.860157    2578 scope.go:117] "RemoveContainer" containerID="3e7335d4efbe1a13d705ff5f704dba67f5a5af5b91d1af9c10799a151bf299e8"
Jul 09 03:15:04.860466 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:15:04.860447    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e7335d4efbe1a13d705ff5f704dba67f5a5af5b91d1af9c10799a151bf299e8\": container with ID starting with 3e7335d4efbe1a13d705ff5f704dba67f5a5af5b91d1af9c10799a151bf299e8 not found: ID does not exist" containerID="3e7335d4efbe1a13d705ff5f704dba67f5a5af5b91d1af9c10799a151bf299e8"
Jul 09 03:15:04.860542 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.860474    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e7335d4efbe1a13d705ff5f704dba67f5a5af5b91d1af9c10799a151bf299e8"} err="failed to get container status \"3e7335d4efbe1a13d705ff5f704dba67f5a5af5b91d1af9c10799a151bf299e8\": rpc error: code = NotFound desc = could not find container \"3e7335d4efbe1a13d705ff5f704dba67f5a5af5b91d1af9c10799a151bf299e8\": container with ID starting with 3e7335d4efbe1a13d705ff5f704dba67f5a5af5b91d1af9c10799a151bf299e8 not found: ID does not exist"
Jul 09 03:15:04.870770 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.870747    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"]
Jul 09 03:15:04.873096 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:04.873074    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cf3835-8484f8674c-5p8bg"]
Jul 09 03:15:06.117651 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:06.117617    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b4e0571-9053-449f-bddc-3d6ba66cea95" path="/var/lib/kubelet/pods/3b4e0571-9053-449f-bddc-3d6ba66cea95/volumes"
Jul 09 03:15:34.542477 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:34.542452    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r"
Jul 09 03:15:34.637265 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:34.637205    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6kzs\" (UniqueName: \"kubernetes.io/projected/96b46041-c5b7-495d-ba2a-d68276259891-kube-api-access-v6kzs\") pod \"96b46041-c5b7-495d-ba2a-d68276259891\" (UID: \"96b46041-c5b7-495d-ba2a-d68276259891\") "
Jul 09 03:15:34.639182 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:34.639148    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96b46041-c5b7-495d-ba2a-d68276259891-kube-api-access-v6kzs" (OuterVolumeSpecName: "kube-api-access-v6kzs") pod "96b46041-c5b7-495d-ba2a-d68276259891" (UID: "96b46041-c5b7-495d-ba2a-d68276259891"). InnerVolumeSpecName "kube-api-access-v6kzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:15:34.738374 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:34.738345    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6kzs\" (UniqueName: \"kubernetes.io/projected/96b46041-c5b7-495d-ba2a-d68276259891-kube-api-access-v6kzs\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:15:35.009602 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:35.009523    2578 generic.go:356] "Generic (PLEG): container finished" podID="96b46041-c5b7-495d-ba2a-d68276259891" containerID="17d1bbaec0384c939e7207aefa530ee5f7e761efedbe2f16a0df564b319c747e" exitCode=137
Jul 09 03:15:35.009602 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:35.009575    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r"
Jul 09 03:15:35.009790 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:35.009627    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r" event={"ID":"96b46041-c5b7-495d-ba2a-d68276259891","Type":"ContainerDied","Data":"17d1bbaec0384c939e7207aefa530ee5f7e761efedbe2f16a0df564b319c747e"}
Jul 09 03:15:35.009790 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:35.009707    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r" event={"ID":"96b46041-c5b7-495d-ba2a-d68276259891","Type":"ContainerDied","Data":"7fa8bb30cef4b4eb9eba3f229e79b885efa5979ae7cb1a4633bf093839293f9c"}
Jul 09 03:15:35.009790 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:35.009741    2578 scope.go:117] "RemoveContainer" containerID="17d1bbaec0384c939e7207aefa530ee5f7e761efedbe2f16a0df564b319c747e"
Jul 09 03:15:35.020098 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:35.020079    2578 scope.go:117] "RemoveContainer" containerID="17d1bbaec0384c939e7207aefa530ee5f7e761efedbe2f16a0df564b319c747e"
Jul 09 03:15:35.020461 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:15:35.020440    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17d1bbaec0384c939e7207aefa530ee5f7e761efedbe2f16a0df564b319c747e\": container with ID starting with 17d1bbaec0384c939e7207aefa530ee5f7e761efedbe2f16a0df564b319c747e not found: ID does not exist" containerID="17d1bbaec0384c939e7207aefa530ee5f7e761efedbe2f16a0df564b319c747e"
Jul 09 03:15:35.020543 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:35.020469    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17d1bbaec0384c939e7207aefa530ee5f7e761efedbe2f16a0df564b319c747e"} err="failed to get container status \"17d1bbaec0384c939e7207aefa530ee5f7e761efedbe2f16a0df564b319c747e\": rpc error: code = NotFound desc = could not find container \"17d1bbaec0384c939e7207aefa530ee5f7e761efedbe2f16a0df564b319c747e\": container with ID starting with 17d1bbaec0384c939e7207aefa530ee5f7e761efedbe2f16a0df564b319c747e not found: ID does not exist"
Jul 09 03:15:35.034950 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:35.034926    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r"]
Jul 09 03:15:35.041040 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:35.041012    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cf3835-29726115-kcz8r"]
Jul 09 03:15:36.117672 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:36.117643    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96b46041-c5b7-495d-ba2a-d68276259891" path="/var/lib/kubelet/pods/96b46041-c5b7-495d-ba2a-d68276259891/volumes"
Jul 09 03:15:43.136447 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:43.136412    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"]
Jul 09 03:15:43.137194 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:43.137161    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg" podUID="d559e056-c5fb-464c-9dcd-f9c18f3741d0" containerName="maas-api" containerID="cri-o://e0105a964da7871017e997cff46f56edcaffc794771b6de8520ee0274de663f5" gracePeriod=30
Jul 09 03:15:43.163468 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:43.163443    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf"]
Jul 09 03:15:43.163700 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:43.163658    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf" podUID="9c863ceb-0117-4e7b-ba06-27253577f37c" containerName="cleanup" containerID="cri-o://ede59db66dc58dafb343299087af8c63681e410d41d8145b73daa3b1404ddb25" gracePeriod=30
Jul 09 03:15:43.395854 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:43.395801    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"
Jul 09 03:15:43.414488 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:43.414455    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkj8d\" (UniqueName: \"kubernetes.io/projected/d559e056-c5fb-464c-9dcd-f9c18f3741d0-kube-api-access-tkj8d\") pod \"d559e056-c5fb-464c-9dcd-f9c18f3741d0\" (UID: \"d559e056-c5fb-464c-9dcd-f9c18f3741d0\") "
Jul 09 03:15:43.414596 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:43.414554    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d559e056-c5fb-464c-9dcd-f9c18f3741d0-openshift-service-ca\") pod \"d559e056-c5fb-464c-9dcd-f9c18f3741d0\" (UID: \"d559e056-c5fb-464c-9dcd-f9c18f3741d0\") "
Jul 09 03:15:43.414663 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:43.414634    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d559e056-c5fb-464c-9dcd-f9c18f3741d0-maas-api-tls\") pod \"d559e056-c5fb-464c-9dcd-f9c18f3741d0\" (UID: \"d559e056-c5fb-464c-9dcd-f9c18f3741d0\") "
Jul 09 03:15:43.415022 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:43.414990    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d559e056-c5fb-464c-9dcd-f9c18f3741d0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d559e056-c5fb-464c-9dcd-f9c18f3741d0" (UID: "d559e056-c5fb-464c-9dcd-f9c18f3741d0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 03:15:43.416995 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:43.416969    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d559e056-c5fb-464c-9dcd-f9c18f3741d0-kube-api-access-tkj8d" (OuterVolumeSpecName: "kube-api-access-tkj8d") pod "d559e056-c5fb-464c-9dcd-f9c18f3741d0" (UID: "d559e056-c5fb-464c-9dcd-f9c18f3741d0"). InnerVolumeSpecName "kube-api-access-tkj8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:15:43.417204 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:43.417174    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d559e056-c5fb-464c-9dcd-f9c18f3741d0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d559e056-c5fb-464c-9dcd-f9c18f3741d0" (UID: "d559e056-c5fb-464c-9dcd-f9c18f3741d0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 03:15:43.515888 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:43.515859    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkj8d\" (UniqueName: \"kubernetes.io/projected/d559e056-c5fb-464c-9dcd-f9c18f3741d0-kube-api-access-tkj8d\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:15:43.515888 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:43.515887    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d559e056-c5fb-464c-9dcd-f9c18f3741d0-openshift-service-ca\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:15:43.516027 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:43.515901    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d559e056-c5fb-464c-9dcd-f9c18f3741d0-maas-api-tls\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:15:44.058630 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:44.058597    2578 generic.go:356] "Generic (PLEG): container finished" podID="d559e056-c5fb-464c-9dcd-f9c18f3741d0" containerID="e0105a964da7871017e997cff46f56edcaffc794771b6de8520ee0274de663f5" exitCode=0
Jul 09 03:15:44.058804 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:44.058645    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"
Jul 09 03:15:44.058804 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:44.058681    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg" event={"ID":"d559e056-c5fb-464c-9dcd-f9c18f3741d0","Type":"ContainerDied","Data":"e0105a964da7871017e997cff46f56edcaffc794771b6de8520ee0274de663f5"}
Jul 09 03:15:44.058804 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:44.058737    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg" event={"ID":"d559e056-c5fb-464c-9dcd-f9c18f3741d0","Type":"ContainerDied","Data":"3f1d79e7a7666e387b944e37056e697b15ae295b7e0dc5a1803fc0b2e7097f15"}
Jul 09 03:15:44.058804 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:44.058761    2578 scope.go:117] "RemoveContainer" containerID="e0105a964da7871017e997cff46f56edcaffc794771b6de8520ee0274de663f5"
Jul 09 03:15:44.070566 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:44.069856    2578 scope.go:117] "RemoveContainer" containerID="e0105a964da7871017e997cff46f56edcaffc794771b6de8520ee0274de663f5"
Jul 09 03:15:44.070904 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:15:44.070882    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0105a964da7871017e997cff46f56edcaffc794771b6de8520ee0274de663f5\": container with ID starting with e0105a964da7871017e997cff46f56edcaffc794771b6de8520ee0274de663f5 not found: ID does not exist" containerID="e0105a964da7871017e997cff46f56edcaffc794771b6de8520ee0274de663f5"
Jul 09 03:15:44.070949 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:44.070913    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0105a964da7871017e997cff46f56edcaffc794771b6de8520ee0274de663f5"} err="failed to get container status \"e0105a964da7871017e997cff46f56edcaffc794771b6de8520ee0274de663f5\": rpc error: code = NotFound desc = could not find container \"e0105a964da7871017e997cff46f56edcaffc794771b6de8520ee0274de663f5\": container with ID starting with e0105a964da7871017e997cff46f56edcaffc794771b6de8520ee0274de663f5 not found: ID does not exist"
Jul 09 03:15:44.079710 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:44.079684    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"]
Jul 09 03:15:44.082913 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:44.082893    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-defa37-7d59b4955b-zfgpg"]
Jul 09 03:15:44.118368 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:44.118339    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d559e056-c5fb-464c-9dcd-f9c18f3741d0" path="/var/lib/kubelet/pods/d559e056-c5fb-464c-9dcd-f9c18f3741d0/volumes"
Jul 09 03:15:49.523811 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:49.523779    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"]
Jul 09 03:15:49.524261 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:49.524032    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj" podUID="29ea98d1-701b-4fb1-9bda-7c1979212fd5" containerName="istio-proxy" containerID="cri-o://0e19dcc32f8e3192a9c308820c62e845281d4855802981afada7a4e3ec35de75" gracePeriod=30
Jul 09 03:15:50.379080 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.379046    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"]
Jul 09 03:15:50.379648 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.379623    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d559e056-c5fb-464c-9dcd-f9c18f3741d0" containerName="maas-api"
Jul 09 03:15:50.379801 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.379762    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d559e056-c5fb-464c-9dcd-f9c18f3741d0" containerName="maas-api"
Jul 09 03:15:50.379880 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.379806    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96b46041-c5b7-495d-ba2a-d68276259891" containerName="cleanup"
Jul 09 03:15:50.379880 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.379816    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="96b46041-c5b7-495d-ba2a-d68276259891" containerName="cleanup"
Jul 09 03:15:50.379880 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.379830    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b4e0571-9053-449f-bddc-3d6ba66cea95" containerName="maas-api"
Jul 09 03:15:50.379880 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.379839    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b4e0571-9053-449f-bddc-3d6ba66cea95" containerName="maas-api"
Jul 09 03:15:50.380084 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.379974    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b4e0571-9053-449f-bddc-3d6ba66cea95" containerName="maas-api"
Jul 09 03:15:50.380084 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.379992    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d559e056-c5fb-464c-9dcd-f9c18f3741d0" containerName="maas-api"
Jul 09 03:15:50.380084 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.380005    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="96b46041-c5b7-495d-ba2a-d68276259891" containerName="cleanup"
Jul 09 03:15:50.384826 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.384805    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.386673 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.386653    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-8d5c89-openshift-default-dockercfg-lrj9x\""
Jul 09 03:15:50.394100 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.394076    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"]
Jul 09 03:15:50.478172 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.478138    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-token\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.478309 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.478179    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ca592ab-6060-4c39-85b5-0345c06f04d5-istiod-ca-cert\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.478309 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.478254    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-podinfo\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.478309 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.478289    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-workload-certs\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.478537 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.478338    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-credential-socket\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.478537 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.478365    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n98xv\" (UniqueName: \"kubernetes.io/projected/9ca592ab-6060-4c39-85b5-0345c06f04d5-kube-api-access-n98xv\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.478537 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.478407    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-workload-socket\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.478537 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.478423    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-envoy\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.478537 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.478495    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-data\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.579287 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.579258    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-token\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.579287 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.579291    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ca592ab-6060-4c39-85b5-0345c06f04d5-istiod-ca-cert\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.579808 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.579341    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-podinfo\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.579808 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.579371    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-workload-certs\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.579808 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.579391    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-credential-socket\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.579808 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.579415    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n98xv\" (UniqueName: \"kubernetes.io/projected/9ca592ab-6060-4c39-85b5-0345c06f04d5-kube-api-access-n98xv\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.579808 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.579459    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-workload-socket\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.579808 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.579483    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-envoy\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.579808 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.579520    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-data\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.580182 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.579828    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-credential-socket\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.580182 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.579913    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-data\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.580182 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.579987    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-workload-socket\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.580182 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.580115    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ca592ab-6060-4c39-85b5-0345c06f04d5-istiod-ca-cert\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.580182 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.580139    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-workload-certs\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.581763 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.581744    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-envoy\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.582154 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.582138    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-podinfo\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.586893 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.586864    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-token\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.586983 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.586956    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n98xv\" (UniqueName: \"kubernetes.io/projected/9ca592ab-6060-4c39-85b5-0345c06f04d5-kube-api-access-n98xv\") pod \"e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") " pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.696190 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.696132    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:50.829026 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.828998    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"]
Jul 09 03:15:50.830751 ip-10-0-129-236 kubenswrapper[2578]: W0709 03:15:50.830727    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca592ab_6060_4c39_85b5_0345c06f04d5.slice/crio-d0d09225914eb1403aaeb0edc28c0f43d2ff6dc31ef946b6373a0a4c64739cc5 WatchSource:0}: Error finding container d0d09225914eb1403aaeb0edc28c0f43d2ff6dc31ef946b6373a0a4c64739cc5: Status 404 returned error can't find the container with id d0d09225914eb1403aaeb0edc28c0f43d2ff6dc31ef946b6373a0a4c64739cc5
Jul 09 03:15:50.833390 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.833356    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 03:15:50.833486 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.833423    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 03:15:50.833486 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:50.833449    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 09 03:15:51.097627 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:51.097540    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm" event={"ID":"9ca592ab-6060-4c39-85b5-0345c06f04d5","Type":"ContainerStarted","Data":"9c9a3e0f24d01fda3a6a069de7b35bdf01b5b7db2b6daf973e7fafcd0988973f"}
Jul 09 03:15:51.097627 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:51.097598    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm" event={"ID":"9ca592ab-6060-4c39-85b5-0345c06f04d5","Type":"ContainerStarted","Data":"d0d09225914eb1403aaeb0edc28c0f43d2ff6dc31ef946b6373a0a4c64739cc5"}
Jul 09 03:15:51.117730 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:51.117674    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm" podStartSLOduration=1.117658915 podStartE2EDuration="1.117658915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:15:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:15:51.113337429 +0000 UTC m=+2465.507140837" watchObservedRunningTime="2026-07-09 03:15:51.117658915 +0000 UTC m=+2465.511462328"
Jul 09 03:15:51.696704 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:51.696674    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:51.701632 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:51.701611    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:52.102244 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:52.102219    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:52.103211 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:52.103194    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:15:54.774246 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.774221    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:15:54.817841 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.817779    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-credential-socket\") pod \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") "
Jul 09 03:15:54.817841 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.817834    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istiod-ca-cert\") pod \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") "
Jul 09 03:15:54.818027 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.817855    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-token\") pod \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") "
Jul 09 03:15:54.818027 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.817896    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-workload-certs\") pod \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") "
Jul 09 03:15:54.818027 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.817918    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-envoy\") pod \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") "
Jul 09 03:15:54.818027 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.817933    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkqhc\" (UniqueName: \"kubernetes.io/projected/29ea98d1-701b-4fb1-9bda-7c1979212fd5-kube-api-access-vkqhc\") pod \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") "
Jul 09 03:15:54.818027 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.817948    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-podinfo\") pod \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") "
Jul 09 03:15:54.818027 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.817984    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-workload-socket\") pod \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") "
Jul 09 03:15:54.818027 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.818002    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-data\") pod \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\" (UID: \"29ea98d1-701b-4fb1-9bda-7c1979212fd5\") "
Jul 09 03:15:54.818403 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.818075    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "29ea98d1-701b-4fb1-9bda-7c1979212fd5" (UID: "29ea98d1-701b-4fb1-9bda-7c1979212fd5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:15:54.818403 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.818223    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "29ea98d1-701b-4fb1-9bda-7c1979212fd5" (UID: "29ea98d1-701b-4fb1-9bda-7c1979212fd5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:15:54.818403 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.818237    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "29ea98d1-701b-4fb1-9bda-7c1979212fd5" (UID: "29ea98d1-701b-4fb1-9bda-7c1979212fd5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:15:54.818403 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.818259    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "29ea98d1-701b-4fb1-9bda-7c1979212fd5" (UID: "29ea98d1-701b-4fb1-9bda-7c1979212fd5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 03:15:54.818626 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.818458    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-credential-socket\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:15:54.818626 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.818477    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istiod-ca-cert\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:15:54.818626 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.818491    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-workload-certs\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:15:54.818626 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.818503    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-workload-socket\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:15:54.819027 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.819001    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-data" (OuterVolumeSpecName: "istio-data") pod "29ea98d1-701b-4fb1-9bda-7c1979212fd5" (UID: "29ea98d1-701b-4fb1-9bda-7c1979212fd5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:15:54.820376 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.820283    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "29ea98d1-701b-4fb1-9bda-7c1979212fd5" (UID: "29ea98d1-701b-4fb1-9bda-7c1979212fd5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 03:15:54.820476 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.820411    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29ea98d1-701b-4fb1-9bda-7c1979212fd5-kube-api-access-vkqhc" (OuterVolumeSpecName: "kube-api-access-vkqhc") pod "29ea98d1-701b-4fb1-9bda-7c1979212fd5" (UID: "29ea98d1-701b-4fb1-9bda-7c1979212fd5"). InnerVolumeSpecName "kube-api-access-vkqhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:15:54.820625 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.820598    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "29ea98d1-701b-4fb1-9bda-7c1979212fd5" (UID: "29ea98d1-701b-4fb1-9bda-7c1979212fd5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:15:54.820625 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.820604    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-token" (OuterVolumeSpecName: "istio-token") pod "29ea98d1-701b-4fb1-9bda-7c1979212fd5" (UID: "29ea98d1-701b-4fb1-9bda-7c1979212fd5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:15:54.919177 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.919154    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-token\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:15:54.919177 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.919173    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-envoy\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:15:54.919310 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.919182    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkqhc\" (UniqueName: \"kubernetes.io/projected/29ea98d1-701b-4fb1-9bda-7c1979212fd5-kube-api-access-vkqhc\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:15:54.919310 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.919191    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-podinfo\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:15:54.919310 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:54.919199    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29ea98d1-701b-4fb1-9bda-7c1979212fd5-istio-data\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:15:55.119424 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:55.119394    2578 generic.go:356] "Generic (PLEG): container finished" podID="29ea98d1-701b-4fb1-9bda-7c1979212fd5" containerID="0e19dcc32f8e3192a9c308820c62e845281d4855802981afada7a4e3ec35de75" exitCode=0
Jul 09 03:15:55.119523 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:55.119447    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"
Jul 09 03:15:55.119523 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:55.119483    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj" event={"ID":"29ea98d1-701b-4fb1-9bda-7c1979212fd5","Type":"ContainerDied","Data":"0e19dcc32f8e3192a9c308820c62e845281d4855802981afada7a4e3ec35de75"}
Jul 09 03:15:55.119594 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:55.119536    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj" event={"ID":"29ea98d1-701b-4fb1-9bda-7c1979212fd5","Type":"ContainerDied","Data":"623d902eab458f81cee6ebff04b19141e826a70a4c81dfdde6a42a92f1613261"}
Jul 09 03:15:55.119594 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:55.119562    2578 scope.go:117] "RemoveContainer" containerID="0e19dcc32f8e3192a9c308820c62e845281d4855802981afada7a4e3ec35de75"
Jul 09 03:15:55.129097 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:55.129081    2578 scope.go:117] "RemoveContainer" containerID="e6eb7fa578c29938a250863697459b8daa037398576ab3181398e9b0a86efb49"
Jul 09 03:15:55.137798 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:55.137708    2578 scope.go:117] "RemoveContainer" containerID="0e19dcc32f8e3192a9c308820c62e845281d4855802981afada7a4e3ec35de75"
Jul 09 03:15:55.138016 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:15:55.137998    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e19dcc32f8e3192a9c308820c62e845281d4855802981afada7a4e3ec35de75\": container with ID starting with 0e19dcc32f8e3192a9c308820c62e845281d4855802981afada7a4e3ec35de75 not found: ID does not exist" containerID="0e19dcc32f8e3192a9c308820c62e845281d4855802981afada7a4e3ec35de75"
Jul 09 03:15:55.138078 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:55.138026    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e19dcc32f8e3192a9c308820c62e845281d4855802981afada7a4e3ec35de75"} err="failed to get container status \"0e19dcc32f8e3192a9c308820c62e845281d4855802981afada7a4e3ec35de75\": rpc error: code = NotFound desc = could not find container \"0e19dcc32f8e3192a9c308820c62e845281d4855802981afada7a4e3ec35de75\": container with ID starting with 0e19dcc32f8e3192a9c308820c62e845281d4855802981afada7a4e3ec35de75 not found: ID does not exist"
Jul 09 03:15:55.138078 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:55.138042    2578 scope.go:117] "RemoveContainer" containerID="e6eb7fa578c29938a250863697459b8daa037398576ab3181398e9b0a86efb49"
Jul 09 03:15:55.138347 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:15:55.138309    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6eb7fa578c29938a250863697459b8daa037398576ab3181398e9b0a86efb49\": container with ID starting with e6eb7fa578c29938a250863697459b8daa037398576ab3181398e9b0a86efb49 not found: ID does not exist" containerID="e6eb7fa578c29938a250863697459b8daa037398576ab3181398e9b0a86efb49"
Jul 09 03:15:55.138423 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:55.138355    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6eb7fa578c29938a250863697459b8daa037398576ab3181398e9b0a86efb49"} err="failed to get container status \"e6eb7fa578c29938a250863697459b8daa037398576ab3181398e9b0a86efb49\": rpc error: code = NotFound desc = could not find container \"e6eb7fa578c29938a250863697459b8daa037398576ab3181398e9b0a86efb49\": container with ID starting with e6eb7fa578c29938a250863697459b8daa037398576ab3181398e9b0a86efb49 not found: ID does not exist"
Jul 09 03:15:55.140456 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:55.140435    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"]
Jul 09 03:15:55.144589 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:55.144570    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-defa37-openshift-default-fdf6b9c9f-wjjgj"]
Jul 09 03:15:56.118958 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:15:56.118928    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29ea98d1-701b-4fb1-9bda-7c1979212fd5" path="/var/lib/kubelet/pods/29ea98d1-701b-4fb1-9bda-7c1979212fd5/volumes"
Jul 09 03:16:00.010111 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:00.010078    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p"]
Jul 09 03:16:00.010524 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:00.010280    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p" podUID="b3cb093f-1c42-4526-9b82-d277f54f1815" containerName="cleanup" containerID="cri-o://7cf18f7ef701c0fe828c0a69a16a6859b41a8c4f09c3b6e1281c2b4106eee0d7" gracePeriod=30
Jul 09 03:16:13.362006 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:13.361982    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf"
Jul 09 03:16:13.478618 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:13.478551    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85tfp\" (UniqueName: \"kubernetes.io/projected/9c863ceb-0117-4e7b-ba06-27253577f37c-kube-api-access-85tfp\") pod \"9c863ceb-0117-4e7b-ba06-27253577f37c\" (UID: \"9c863ceb-0117-4e7b-ba06-27253577f37c\") "
Jul 09 03:16:13.480484 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:13.480459    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c863ceb-0117-4e7b-ba06-27253577f37c-kube-api-access-85tfp" (OuterVolumeSpecName: "kube-api-access-85tfp") pod "9c863ceb-0117-4e7b-ba06-27253577f37c" (UID: "9c863ceb-0117-4e7b-ba06-27253577f37c"). InnerVolumeSpecName "kube-api-access-85tfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:16:13.579956 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:13.579933    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85tfp\" (UniqueName: \"kubernetes.io/projected/9c863ceb-0117-4e7b-ba06-27253577f37c-kube-api-access-85tfp\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:16:14.212596 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:14.212561    2578 generic.go:356] "Generic (PLEG): container finished" podID="9c863ceb-0117-4e7b-ba06-27253577f37c" containerID="ede59db66dc58dafb343299087af8c63681e410d41d8145b73daa3b1404ddb25" exitCode=137
Jul 09 03:16:14.212765 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:14.212608    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf"
Jul 09 03:16:14.212765 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:14.212648    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf" event={"ID":"9c863ceb-0117-4e7b-ba06-27253577f37c","Type":"ContainerDied","Data":"ede59db66dc58dafb343299087af8c63681e410d41d8145b73daa3b1404ddb25"}
Jul 09 03:16:14.212765 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:14.212702    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf" event={"ID":"9c863ceb-0117-4e7b-ba06-27253577f37c","Type":"ContainerDied","Data":"10bca3ef53ecc2f45ba464947f28106e2e2f7c5d1b0fbd0fe3c0b0683630dc72"}
Jul 09 03:16:14.212765 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:14.212725    2578 scope.go:117] "RemoveContainer" containerID="ede59db66dc58dafb343299087af8c63681e410d41d8145b73daa3b1404ddb25"
Jul 09 03:16:14.221258 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:14.221241    2578 scope.go:117] "RemoveContainer" containerID="ede59db66dc58dafb343299087af8c63681e410d41d8145b73daa3b1404ddb25"
Jul 09 03:16:14.221550 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:16:14.221530    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ede59db66dc58dafb343299087af8c63681e410d41d8145b73daa3b1404ddb25\": container with ID starting with ede59db66dc58dafb343299087af8c63681e410d41d8145b73daa3b1404ddb25 not found: ID does not exist" containerID="ede59db66dc58dafb343299087af8c63681e410d41d8145b73daa3b1404ddb25"
Jul 09 03:16:14.221619 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:14.221557    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ede59db66dc58dafb343299087af8c63681e410d41d8145b73daa3b1404ddb25"} err="failed to get container status \"ede59db66dc58dafb343299087af8c63681e410d41d8145b73daa3b1404ddb25\": rpc error: code = NotFound desc = could not find container \"ede59db66dc58dafb343299087af8c63681e410d41d8145b73daa3b1404ddb25\": container with ID starting with ede59db66dc58dafb343299087af8c63681e410d41d8145b73daa3b1404ddb25 not found: ID does not exist"
Jul 09 03:16:14.227646 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:14.227624    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf"]
Jul 09 03:16:14.233184 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:14.233163    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-defa37-29726115-8hgxf"]
Jul 09 03:16:16.118671 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:16.118624    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c863ceb-0117-4e7b-ba06-27253577f37c" path="/var/lib/kubelet/pods/9c863ceb-0117-4e7b-ba06-27253577f37c/volumes"
Jul 09 03:16:30.203643 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:30.203614    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p"
Jul 09 03:16:30.291640 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:30.291573    2578 generic.go:356] "Generic (PLEG): container finished" podID="b3cb093f-1c42-4526-9b82-d277f54f1815" containerID="7cf18f7ef701c0fe828c0a69a16a6859b41a8c4f09c3b6e1281c2b4106eee0d7" exitCode=137
Jul 09 03:16:30.291640 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:30.291612    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p"
Jul 09 03:16:30.291640 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:30.291619    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p" event={"ID":"b3cb093f-1c42-4526-9b82-d277f54f1815","Type":"ContainerDied","Data":"7cf18f7ef701c0fe828c0a69a16a6859b41a8c4f09c3b6e1281c2b4106eee0d7"}
Jul 09 03:16:30.291822 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:30.291660    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p" event={"ID":"b3cb093f-1c42-4526-9b82-d277f54f1815","Type":"ContainerDied","Data":"848228d53fa08010131d31961ad03e58a75ff9a44e2797cc3da673282f58c93a"}
Jul 09 03:16:30.291822 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:30.291684    2578 scope.go:117] "RemoveContainer" containerID="7cf18f7ef701c0fe828c0a69a16a6859b41a8c4f09c3b6e1281c2b4106eee0d7"
Jul 09 03:16:30.299783 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:30.299765    2578 scope.go:117] "RemoveContainer" containerID="7cf18f7ef701c0fe828c0a69a16a6859b41a8c4f09c3b6e1281c2b4106eee0d7"
Jul 09 03:16:30.300042 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:16:30.300026    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cf18f7ef701c0fe828c0a69a16a6859b41a8c4f09c3b6e1281c2b4106eee0d7\": container with ID starting with 7cf18f7ef701c0fe828c0a69a16a6859b41a8c4f09c3b6e1281c2b4106eee0d7 not found: ID does not exist" containerID="7cf18f7ef701c0fe828c0a69a16a6859b41a8c4f09c3b6e1281c2b4106eee0d7"
Jul 09 03:16:30.300087 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:30.300053    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cf18f7ef701c0fe828c0a69a16a6859b41a8c4f09c3b6e1281c2b4106eee0d7"} err="failed to get container status \"7cf18f7ef701c0fe828c0a69a16a6859b41a8c4f09c3b6e1281c2b4106eee0d7\": rpc error: code = NotFound desc = could not find container \"7cf18f7ef701c0fe828c0a69a16a6859b41a8c4f09c3b6e1281c2b4106eee0d7\": container with ID starting with 7cf18f7ef701c0fe828c0a69a16a6859b41a8c4f09c3b6e1281c2b4106eee0d7 not found: ID does not exist"
Jul 09 03:16:30.311462 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:30.311442    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7q2r\" (UniqueName: \"kubernetes.io/projected/b3cb093f-1c42-4526-9b82-d277f54f1815-kube-api-access-c7q2r\") pod \"b3cb093f-1c42-4526-9b82-d277f54f1815\" (UID: \"b3cb093f-1c42-4526-9b82-d277f54f1815\") "
Jul 09 03:16:30.313331 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:30.313292    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3cb093f-1c42-4526-9b82-d277f54f1815-kube-api-access-c7q2r" (OuterVolumeSpecName: "kube-api-access-c7q2r") pod "b3cb093f-1c42-4526-9b82-d277f54f1815" (UID: "b3cb093f-1c42-4526-9b82-d277f54f1815"). InnerVolumeSpecName "kube-api-access-c7q2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:16:30.412033 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:30.412008    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7q2r\" (UniqueName: \"kubernetes.io/projected/b3cb093f-1c42-4526-9b82-d277f54f1815-kube-api-access-c7q2r\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:16:30.611696 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:30.611672    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p"]
Jul 09 03:16:30.615311 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:30.615286    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726115-n6z8p"]
Jul 09 03:16:32.118149 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:16:32.118107    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3cb093f-1c42-4526-9b82-d277f54f1815" path="/var/lib/kubelet/pods/b3cb093f-1c42-4526-9b82-d277f54f1815/volumes"
Jul 09 03:19:08.817480 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:08.817444    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"]
Jul 09 03:19:08.820371 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:08.818489    2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm" podUID="9ca592ab-6060-4c39-85b5-0345c06f04d5" containerName="istio-proxy" containerID="cri-o://9c9a3e0f24d01fda3a6a069de7b35bdf01b5b7db2b6daf973e7fafcd0988973f" gracePeriod=30
Jul 09 03:19:14.069878 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.069854    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:19:14.102259 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.102237    2578 generic.go:356] "Generic (PLEG): container finished" podID="9ca592ab-6060-4c39-85b5-0345c06f04d5" containerID="9c9a3e0f24d01fda3a6a069de7b35bdf01b5b7db2b6daf973e7fafcd0988973f" exitCode=0
Jul 09 03:19:14.102408 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.102288    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm" event={"ID":"9ca592ab-6060-4c39-85b5-0345c06f04d5","Type":"ContainerDied","Data":"9c9a3e0f24d01fda3a6a069de7b35bdf01b5b7db2b6daf973e7fafcd0988973f"}
Jul 09 03:19:14.102408 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.102296    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"
Jul 09 03:19:14.102408 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.102379    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm" event={"ID":"9ca592ab-6060-4c39-85b5-0345c06f04d5","Type":"ContainerDied","Data":"d0d09225914eb1403aaeb0edc28c0f43d2ff6dc31ef946b6373a0a4c64739cc5"}
Jul 09 03:19:14.102408 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.102403    2578 scope.go:117] "RemoveContainer" containerID="9c9a3e0f24d01fda3a6a069de7b35bdf01b5b7db2b6daf973e7fafcd0988973f"
Jul 09 03:19:14.110818 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.110798    2578 scope.go:117] "RemoveContainer" containerID="9c9a3e0f24d01fda3a6a069de7b35bdf01b5b7db2b6daf973e7fafcd0988973f"
Jul 09 03:19:14.111076 ip-10-0-129-236 kubenswrapper[2578]: E0709 03:19:14.111057    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c9a3e0f24d01fda3a6a069de7b35bdf01b5b7db2b6daf973e7fafcd0988973f\": container with ID starting with 9c9a3e0f24d01fda3a6a069de7b35bdf01b5b7db2b6daf973e7fafcd0988973f not found: ID does not exist" containerID="9c9a3e0f24d01fda3a6a069de7b35bdf01b5b7db2b6daf973e7fafcd0988973f"
Jul 09 03:19:14.111126 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.111085    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c9a3e0f24d01fda3a6a069de7b35bdf01b5b7db2b6daf973e7fafcd0988973f"} err="failed to get container status \"9c9a3e0f24d01fda3a6a069de7b35bdf01b5b7db2b6daf973e7fafcd0988973f\": rpc error: code = NotFound desc = could not find container \"9c9a3e0f24d01fda3a6a069de7b35bdf01b5b7db2b6daf973e7fafcd0988973f\": container with ID starting with 9c9a3e0f24d01fda3a6a069de7b35bdf01b5b7db2b6daf973e7fafcd0988973f not found: ID does not exist"
Jul 09 03:19:14.160927 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.160908    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n98xv\" (UniqueName: \"kubernetes.io/projected/9ca592ab-6060-4c39-85b5-0345c06f04d5-kube-api-access-n98xv\") pod \"9ca592ab-6060-4c39-85b5-0345c06f04d5\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") "
Jul 09 03:19:14.161042 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.160936    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-podinfo\") pod \"9ca592ab-6060-4c39-85b5-0345c06f04d5\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") "
Jul 09 03:19:14.161042 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.160985    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-data\") pod \"9ca592ab-6060-4c39-85b5-0345c06f04d5\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") "
Jul 09 03:19:14.161042 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.161002    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-envoy\") pod \"9ca592ab-6060-4c39-85b5-0345c06f04d5\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") "
Jul 09 03:19:14.161042 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.161021    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-workload-certs\") pod \"9ca592ab-6060-4c39-85b5-0345c06f04d5\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") "
Jul 09 03:19:14.161042 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.161043    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-credential-socket\") pod \"9ca592ab-6060-4c39-85b5-0345c06f04d5\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") "
Jul 09 03:19:14.161308 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.161093    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ca592ab-6060-4c39-85b5-0345c06f04d5-istiod-ca-cert\") pod \"9ca592ab-6060-4c39-85b5-0345c06f04d5\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") "
Jul 09 03:19:14.161308 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.161113    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-token\") pod \"9ca592ab-6060-4c39-85b5-0345c06f04d5\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") "
Jul 09 03:19:14.161308 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.161138    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-workload-socket\") pod \"9ca592ab-6060-4c39-85b5-0345c06f04d5\" (UID: \"9ca592ab-6060-4c39-85b5-0345c06f04d5\") "
Jul 09 03:19:14.161308 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.161268    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-data" (OuterVolumeSpecName: "istio-data") pod "9ca592ab-6060-4c39-85b5-0345c06f04d5" (UID: "9ca592ab-6060-4c39-85b5-0345c06f04d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:19:14.161308 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.161287    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9ca592ab-6060-4c39-85b5-0345c06f04d5" (UID: "9ca592ab-6060-4c39-85b5-0345c06f04d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:19:14.161615 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.161466    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9ca592ab-6060-4c39-85b5-0345c06f04d5" (UID: "9ca592ab-6060-4c39-85b5-0345c06f04d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:19:14.161615 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.161510    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ca592ab-6060-4c39-85b5-0345c06f04d5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9ca592ab-6060-4c39-85b5-0345c06f04d5" (UID: "9ca592ab-6060-4c39-85b5-0345c06f04d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 03:19:14.161615 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.161585    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-data\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:19:14.161780 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.161623    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-workload-certs\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:19:14.161780 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.161637    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ca592ab-6060-4c39-85b5-0345c06f04d5-istiod-ca-cert\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:19:14.161780 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.161650    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-workload-socket\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:19:14.161780 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.161677    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9ca592ab-6060-4c39-85b5-0345c06f04d5" (UID: "9ca592ab-6060-4c39-85b5-0345c06f04d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:19:14.163069 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.163027    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ca592ab-6060-4c39-85b5-0345c06f04d5-kube-api-access-n98xv" (OuterVolumeSpecName: "kube-api-access-n98xv") pod "9ca592ab-6060-4c39-85b5-0345c06f04d5" (UID: "9ca592ab-6060-4c39-85b5-0345c06f04d5"). InnerVolumeSpecName "kube-api-access-n98xv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:19:14.163176 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.163126    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9ca592ab-6060-4c39-85b5-0345c06f04d5" (UID: "9ca592ab-6060-4c39-85b5-0345c06f04d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 03:19:14.163466 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.163442    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9ca592ab-6060-4c39-85b5-0345c06f04d5" (UID: "9ca592ab-6060-4c39-85b5-0345c06f04d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 03:19:14.163519 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.163485    2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-token" (OuterVolumeSpecName: "istio-token") pod "9ca592ab-6060-4c39-85b5-0345c06f04d5" (UID: "9ca592ab-6060-4c39-85b5-0345c06f04d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 03:19:14.262478 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.262455    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-envoy\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:19:14.262478 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.262479    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca592ab-6060-4c39-85b5-0345c06f04d5-credential-socket\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:19:14.262630 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.262488    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-token\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:19:14.262630 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.262499    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n98xv\" (UniqueName: \"kubernetes.io/projected/9ca592ab-6060-4c39-85b5-0345c06f04d5-kube-api-access-n98xv\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:19:14.262630 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.262510    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ca592ab-6060-4c39-85b5-0345c06f04d5-istio-podinfo\") on node \"ip-10-0-129-236.ec2.internal\" DevicePath \"\""
Jul 09 03:19:14.423083 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.423052    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"]
Jul 09 03:19:14.426580 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:14.426558    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-8d5c89-openshift-default-59b8cc5ffc-7kqsm"]
Jul 09 03:19:15.141354 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:15.141327    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5fc988df-dkq9d_42b4081d-2fb4-422a-acc5-2e707eae8a94/manager/0.log"
Jul 09 03:19:15.724890 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:15.724858    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75df5ddb9c-m5mtm_1e0e49f3-2077-43ad-b073-a2d38a355d40/copy-manifests/0.log"
Jul 09 03:19:15.731778 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:15.731753    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75df5ddb9c-m5mtm_1e0e49f3-2077-43ad-b073-a2d38a355d40/manager/0.log"
Jul 09 03:19:16.118045 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:16.118012    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ca592ab-6060-4c39-85b5-0345c06f04d5" path="/var/lib/kubelet/pods/9ca592ab-6060-4c39-85b5-0345c06f04d5/volumes"
Jul 09 03:19:17.089854 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:17.089819    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-jkb7l_82d746e1-23a4-4986-8afd-1d6265b4ebc1/manager/0.log"
Jul 09 03:19:17.199797 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:17.199769    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-77k6w_df551de3-1753-4995-91f3-8198f0071445/manager/0.log"
Jul 09 03:19:17.428924 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:17.428897    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-htmvv_8d341bd6-f964-4887-a678-b50dd7788546/registry-server/0.log"
Jul 09 03:19:17.551593 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:17.551565    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-g496q_c9711e01-33c9-4967-8001-bbe3c904f253/manager/0.log"
Jul 09 03:19:17.798548 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:17.798478    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-lhqz8_f4ebfaeb-0956-4f01-8f42-c4c5cf21e52d/manager/0.log"
Jul 09 03:19:18.270557 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:18.270532    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2zf89_041897a1-32d2-4316-8f58-2b6fe74e92f9/discovery/0.log"
Jul 09 03:19:18.483991 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:18.483966    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75465c7b46-h57n5_da8b0328-7d6f-497b-a0df-8eec6c05632d/kube-auth-proxy/0.log"
Jul 09 03:19:18.721282 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:18.721254    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-k5674_d59f60c5-7096-4144-bb66-41770698c788/payload-pre-processing/0.log"
Jul 09 03:19:18.842734 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:18.842700    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-s4mtn_a4dc21f4-39f2-4826-927e-eda9227e102f/payload-processing/0.log"
Jul 09 03:19:19.398270 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:19.398233    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-5xr2w_dccd6202-276b-445d-8853-900bd0daa99c/main/0.log"
Jul 09 03:19:19.614916 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:19.614876    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbgqjwm_50b17a53-46ea-4987-b4ae-7cd74cfa05f0/main/0.log"
Jul 09 03:19:26.741284 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:26.741257    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-t6lpr_759cd68b-755a-47d5-bb83-7a9028adbc86/global-pull-secret-syncer/0.log"
Jul 09 03:19:26.900212 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:26.900182    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pzqxx_1508f1fe-46da-4bfa-bbd0-a975c19cf63b/konnectivity-agent/0.log"
Jul 09 03:19:26.928126 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:26.928103    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-236.ec2.internal_4d17de08da950bb8f59f5515256e0343/haproxy/0.log"
Jul 09 03:19:32.440761 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:32.440730    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-jkb7l_82d746e1-23a4-4986-8afd-1d6265b4ebc1/manager/0.log"
Jul 09 03:19:32.461839 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:32.461818    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-77k6w_df551de3-1753-4995-91f3-8198f0071445/manager/0.log"
Jul 09 03:19:32.535429 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:32.535410    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-htmvv_8d341bd6-f964-4887-a678-b50dd7788546/registry-server/0.log"
Jul 09 03:19:32.674102 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:32.674074    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-g496q_c9711e01-33c9-4967-8001-bbe3c904f253/manager/0.log"
Jul 09 03:19:32.783241 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:32.783176    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-lhqz8_f4ebfaeb-0956-4f01-8f42-c4c5cf21e52d/manager/0.log"
Jul 09 03:19:34.742831 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:34.742790    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-zlnbl_6e15e0a7-3794-464b-8af7-87ec9972b022/kube-state-metrics/0.log"
Jul 09 03:19:34.771420 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:34.771357    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-zlnbl_6e15e0a7-3794-464b-8af7-87ec9972b022/kube-rbac-proxy-main/0.log"
Jul 09 03:19:34.794586 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:34.794564    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-zlnbl_6e15e0a7-3794-464b-8af7-87ec9972b022/kube-rbac-proxy-self/0.log"
Jul 09 03:19:34.820563 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:34.820541    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7f7b4c4ff5-qfsxt_a23f1777-3ef1-42e8-9078-ff2a3176fda1/metrics-server/0.log"
Jul 09 03:19:35.028245 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.028179    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kplfz_2c5d0295-77c6-4ea3-9826-a7978bbfb31c/node-exporter/0.log"
Jul 09 03:19:35.052114 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.052094    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kplfz_2c5d0295-77c6-4ea3-9826-a7978bbfb31c/kube-rbac-proxy/0.log"
Jul 09 03:19:35.074354 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.074338    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kplfz_2c5d0295-77c6-4ea3-9826-a7978bbfb31c/init-textfile/0.log"
Jul 09 03:19:35.105453 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.105434    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-c5872_47f5a573-8ada-41c9-b55d-67c511698be5/kube-rbac-proxy-main/0.log"
Jul 09 03:19:35.128341 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.128294    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-c5872_47f5a573-8ada-41c9-b55d-67c511698be5/kube-rbac-proxy-self/0.log"
Jul 09 03:19:35.148552 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.148531    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-c5872_47f5a573-8ada-41c9-b55d-67c511698be5/openshift-state-metrics/0.log"
Jul 09 03:19:35.184155 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.184133    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_881efdbb-3e3c-456b-b0cc-254642d3719f/prometheus/0.log"
Jul 09 03:19:35.208480 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.208463    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_881efdbb-3e3c-456b-b0cc-254642d3719f/config-reloader/0.log"
Jul 09 03:19:35.229707 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.229687    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_881efdbb-3e3c-456b-b0cc-254642d3719f/thanos-sidecar/0.log"
Jul 09 03:19:35.249530 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.249509    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_881efdbb-3e3c-456b-b0cc-254642d3719f/kube-rbac-proxy-web/0.log"
Jul 09 03:19:35.269572 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.269554    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_881efdbb-3e3c-456b-b0cc-254642d3719f/kube-rbac-proxy/0.log"
Jul 09 03:19:35.290364 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.290293    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_881efdbb-3e3c-456b-b0cc-254642d3719f/kube-rbac-proxy-thanos/0.log"
Jul 09 03:19:35.313451 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.313437    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_881efdbb-3e3c-456b-b0cc-254642d3719f/init-config-reloader/0.log"
Jul 09 03:19:35.431562 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.431540    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c7d6df564-rhbv6_25533b64-b04c-4912-96ec-caa77912b43e/telemeter-client/0.log"
Jul 09 03:19:35.453785 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.453768    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c7d6df564-rhbv6_25533b64-b04c-4912-96ec-caa77912b43e/reload/0.log"
Jul 09 03:19:35.474061 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.474041    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c7d6df564-rhbv6_25533b64-b04c-4912-96ec-caa77912b43e/kube-rbac-proxy/0.log"
Jul 09 03:19:35.503846 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.503825    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66fd8c76dd-t2grq_542f470c-85ec-4d82-9064-ff81b0b3177e/thanos-query/0.log"
Jul 09 03:19:35.528991 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.528969    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66fd8c76dd-t2grq_542f470c-85ec-4d82-9064-ff81b0b3177e/kube-rbac-proxy-web/0.log"
Jul 09 03:19:35.555226 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.555166    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66fd8c76dd-t2grq_542f470c-85ec-4d82-9064-ff81b0b3177e/kube-rbac-proxy/0.log"
Jul 09 03:19:35.574514 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.574498    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66fd8c76dd-t2grq_542f470c-85ec-4d82-9064-ff81b0b3177e/prom-label-proxy/0.log"
Jul 09 03:19:35.595243 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.595223    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66fd8c76dd-t2grq_542f470c-85ec-4d82-9064-ff81b0b3177e/kube-rbac-proxy-rules/0.log"
Jul 09 03:19:35.616348 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:35.616307    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66fd8c76dd-t2grq_542f470c-85ec-4d82-9064-ff81b0b3177e/kube-rbac-proxy-metrics/0.log"
Jul 09 03:19:36.920936 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.920899    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"]
Jul 09 03:19:36.921515 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.921489    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ca592ab-6060-4c39-85b5-0345c06f04d5" containerName="istio-proxy"
Jul 09 03:19:36.921515 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.921512    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ca592ab-6060-4c39-85b5-0345c06f04d5" containerName="istio-proxy"
Jul 09 03:19:36.921515 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.921525    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29ea98d1-701b-4fb1-9bda-7c1979212fd5" containerName="istio-proxy"
Jul 09 03:19:36.921515 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.921531    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="29ea98d1-701b-4fb1-9bda-7c1979212fd5" containerName="istio-proxy"
Jul 09 03:19:36.921855 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.921540    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3cb093f-1c42-4526-9b82-d277f54f1815" containerName="cleanup"
Jul 09 03:19:36.921855 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.921549    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3cb093f-1c42-4526-9b82-d277f54f1815" containerName="cleanup"
Jul 09 03:19:36.921855 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.921565    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29ea98d1-701b-4fb1-9bda-7c1979212fd5" containerName="istio-proxy"
Jul 09 03:19:36.921855 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.921573    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="29ea98d1-701b-4fb1-9bda-7c1979212fd5" containerName="istio-proxy"
Jul 09 03:19:36.921855 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.921582    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c863ceb-0117-4e7b-ba06-27253577f37c" containerName="cleanup"
Jul 09 03:19:36.921855 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.921591    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c863ceb-0117-4e7b-ba06-27253577f37c" containerName="cleanup"
Jul 09 03:19:36.921855 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.921703    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c863ceb-0117-4e7b-ba06-27253577f37c" containerName="cleanup"
Jul 09 03:19:36.921855 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.921719    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="29ea98d1-701b-4fb1-9bda-7c1979212fd5" containerName="istio-proxy"
Jul 09 03:19:36.921855 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.921731    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="29ea98d1-701b-4fb1-9bda-7c1979212fd5" containerName="istio-proxy"
Jul 09 03:19:36.921855 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.921741    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3cb093f-1c42-4526-9b82-d277f54f1815" containerName="cleanup"
Jul 09 03:19:36.921855 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.921751    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ca592ab-6060-4c39-85b5-0345c06f04d5" containerName="istio-proxy"
Jul 09 03:19:36.925267 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.925251    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:36.926909 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.926890    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9jjkg\"/\"kube-root-ca.crt\""
Jul 09 03:19:36.927359 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.927340    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9jjkg\"/\"openshift-service-ca.crt\""
Jul 09 03:19:36.927422 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.927340    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9jjkg\"/\"default-dockercfg-84tvq\""
Jul 09 03:19:36.933131 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:36.933113    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"]
Jul 09 03:19:37.043970 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.043944    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a51d9d8d-f15c-4503-9947-9528ed893162-proc\") pod \"perf-node-gather-daemonset-hgfgp\" (UID: \"a51d9d8d-f15c-4503-9947-9528ed893162\") " pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:37.044103 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.043978    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a51d9d8d-f15c-4503-9947-9528ed893162-sys\") pod \"perf-node-gather-daemonset-hgfgp\" (UID: \"a51d9d8d-f15c-4503-9947-9528ed893162\") " pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:37.044103 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.043999    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a51d9d8d-f15c-4503-9947-9528ed893162-lib-modules\") pod \"perf-node-gather-daemonset-hgfgp\" (UID: \"a51d9d8d-f15c-4503-9947-9528ed893162\") " pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:37.044222 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.044093    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a51d9d8d-f15c-4503-9947-9528ed893162-podres\") pod \"perf-node-gather-daemonset-hgfgp\" (UID: \"a51d9d8d-f15c-4503-9947-9528ed893162\") " pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:37.044222 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.044170    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2xch\" (UniqueName: \"kubernetes.io/projected/a51d9d8d-f15c-4503-9947-9528ed893162-kube-api-access-t2xch\") pod \"perf-node-gather-daemonset-hgfgp\" (UID: \"a51d9d8d-f15c-4503-9947-9528ed893162\") " pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:37.145196 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.145171    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2xch\" (UniqueName: \"kubernetes.io/projected/a51d9d8d-f15c-4503-9947-9528ed893162-kube-api-access-t2xch\") pod \"perf-node-gather-daemonset-hgfgp\" (UID: \"a51d9d8d-f15c-4503-9947-9528ed893162\") " pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:37.145301 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.145230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a51d9d8d-f15c-4503-9947-9528ed893162-proc\") pod \"perf-node-gather-daemonset-hgfgp\" (UID: \"a51d9d8d-f15c-4503-9947-9528ed893162\") " pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:37.145301 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.145263    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a51d9d8d-f15c-4503-9947-9528ed893162-sys\") pod \"perf-node-gather-daemonset-hgfgp\" (UID: \"a51d9d8d-f15c-4503-9947-9528ed893162\") " pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:37.145301 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.145292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a51d9d8d-f15c-4503-9947-9528ed893162-lib-modules\") pod \"perf-node-gather-daemonset-hgfgp\" (UID: \"a51d9d8d-f15c-4503-9947-9528ed893162\") " pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:37.145495 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.145403    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a51d9d8d-f15c-4503-9947-9528ed893162-proc\") pod \"perf-node-gather-daemonset-hgfgp\" (UID: \"a51d9d8d-f15c-4503-9947-9528ed893162\") " pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:37.145495 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.145427    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a51d9d8d-f15c-4503-9947-9528ed893162-podres\") pod \"perf-node-gather-daemonset-hgfgp\" (UID: \"a51d9d8d-f15c-4503-9947-9528ed893162\") " pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:37.145495 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.145408    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a51d9d8d-f15c-4503-9947-9528ed893162-sys\") pod \"perf-node-gather-daemonset-hgfgp\" (UID: \"a51d9d8d-f15c-4503-9947-9528ed893162\") " pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:37.145495 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.145457    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a51d9d8d-f15c-4503-9947-9528ed893162-lib-modules\") pod \"perf-node-gather-daemonset-hgfgp\" (UID: \"a51d9d8d-f15c-4503-9947-9528ed893162\") " pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:37.145640 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.145501    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a51d9d8d-f15c-4503-9947-9528ed893162-podres\") pod \"perf-node-gather-daemonset-hgfgp\" (UID: \"a51d9d8d-f15c-4503-9947-9528ed893162\") " pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:37.153641 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.153623    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2xch\" (UniqueName: \"kubernetes.io/projected/a51d9d8d-f15c-4503-9947-9528ed893162-kube-api-access-t2xch\") pod \"perf-node-gather-daemonset-hgfgp\" (UID: \"a51d9d8d-f15c-4503-9947-9528ed893162\") " pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:37.238190 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.238138    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:37.363387 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:37.363355    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"]
Jul 09 03:19:37.366949 ip-10-0-129-236 kubenswrapper[2578]: W0709 03:19:37.366922    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda51d9d8d_f15c_4503_9947_9528ed893162.slice/crio-734b1525d6928725e6b7be6a9c6554074cd1865b9193bb67099e6a64f68461b9 WatchSource:0}: Error finding container 734b1525d6928725e6b7be6a9c6554074cd1865b9193bb67099e6a64f68461b9: Status 404 returned error can't find the container with id 734b1525d6928725e6b7be6a9c6554074cd1865b9193bb67099e6a64f68461b9
Jul 09 03:19:38.219634 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:38.219549    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp" event={"ID":"a51d9d8d-f15c-4503-9947-9528ed893162","Type":"ContainerStarted","Data":"b36224865fee95f6169075a8f88b2cdc9a6dcf10af32b00476477c65cbc5e28b"}
Jul 09 03:19:38.219634 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:38.219600    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp" event={"ID":"a51d9d8d-f15c-4503-9947-9528ed893162","Type":"ContainerStarted","Data":"734b1525d6928725e6b7be6a9c6554074cd1865b9193bb67099e6a64f68461b9"}
Jul 09 03:19:38.220083 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:38.219635    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:38.234392 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:38.234344    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp" podStartSLOduration=2.234329682 podStartE2EDuration="2.234329682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:19:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 03:19:38.233054139 +0000 UTC m=+2692.626857552" watchObservedRunningTime="2026-07-09 03:19:38.234329682 +0000 UTC m=+2692.628133098"
Jul 09 03:19:39.570284 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:39.570258    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j99cn_d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa/dns/0.log"
Jul 09 03:19:39.590012 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:39.589990    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j99cn_d7b28cf3-0900-48a4-a3a2-27e4ef3d72aa/kube-rbac-proxy/0.log"
Jul 09 03:19:39.703466 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:39.703445    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4rnpt_daefa1ce-31aa-4af4-8ec5-6a5b969a2dc6/dns-node-resolver/0.log"
Jul 09 03:19:40.323786 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:40.323759    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kxrl5_147eafc1-fac7-4201-b3d1-261371a5667a/node-ca/0.log"
Jul 09 03:19:41.415668 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:41.415632    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2zf89_041897a1-32d2-4316-8f58-2b6fe74e92f9/discovery/0.log"
Jul 09 03:19:41.457207 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:41.457178    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75465c7b46-h57n5_da8b0328-7d6f-497b-a0df-8eec6c05632d/kube-auth-proxy/0.log"
Jul 09 03:19:41.571665 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:41.571635    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-k5674_d59f60c5-7096-4144-bb66-41770698c788/payload-pre-processing/0.log"
Jul 09 03:19:41.640254 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:41.640234    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-s4mtn_a4dc21f4-39f2-4826-927e-eda9227e102f/payload-processing/0.log"
Jul 09 03:19:42.312141 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:42.312114    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6b29q_c2415615-0367-4de5-b7f8-afd1945d4b4e/serve-healthcheck-canary/0.log"
Jul 09 03:19:42.992823 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:42.992794    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zc8cn_9b76c7f8-c7e2-48dd-a392-90409477fb13/kube-rbac-proxy/0.log"
Jul 09 03:19:43.012512 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:43.012480    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zc8cn_9b76c7f8-c7e2-48dd-a392-90409477fb13/exporter/0.log"
Jul 09 03:19:43.033200 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:43.033163    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zc8cn_9b76c7f8-c7e2-48dd-a392-90409477fb13/extractor/0.log"
Jul 09 03:19:44.234731 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:44.234709    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9jjkg/perf-node-gather-daemonset-hgfgp"
Jul 09 03:19:45.384684 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:45.384651    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c5fc988df-dkq9d_42b4081d-2fb4-422a-acc5-2e707eae8a94/manager/0.log"
Jul 09 03:19:45.679576 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:45.679506    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75df5ddb9c-m5mtm_1e0e49f3-2077-43ad-b073-a2d38a355d40/manager/0.log"
Jul 09 03:19:45.697173 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:45.697151    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75df5ddb9c-m5mtm_1e0e49f3-2077-43ad-b073-a2d38a355d40/copy-manifests/0.log"
Jul 09 03:19:48.335957 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:48.335925    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2zf89_041897a1-32d2-4316-8f58-2b6fe74e92f9/discovery/0.log"
Jul 09 03:19:48.361816 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:48.361781    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75465c7b46-h57n5_da8b0328-7d6f-497b-a0df-8eec6c05632d/kube-auth-proxy/0.log"
Jul 09 03:19:48.464550 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:48.464490    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-k5674_d59f60c5-7096-4144-bb66-41770698c788/payload-pre-processing/0.log"
Jul 09 03:19:48.526211 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:48.526184    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-s4mtn_a4dc21f4-39f2-4826-927e-eda9227e102f/payload-processing/0.log"
Jul 09 03:19:49.811129 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:49.811099    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2zf89_041897a1-32d2-4316-8f58-2b6fe74e92f9/discovery/0.log"
Jul 09 03:19:49.839533 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:49.839509    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75465c7b46-h57n5_da8b0328-7d6f-497b-a0df-8eec6c05632d/kube-auth-proxy/0.log"
Jul 09 03:19:49.933564 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:49.933544    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-k5674_d59f60c5-7096-4144-bb66-41770698c788/payload-pre-processing/0.log"
Jul 09 03:19:49.997560 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:49.997532    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-s4mtn_a4dc21f4-39f2-4826-927e-eda9227e102f/payload-processing/0.log"
Jul 09 03:19:53.629854 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:53.629821    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5jtrv_77446bdb-57d3-4db7-8518-63cdb3111bd9/kube-multus/0.log"
Jul 09 03:19:53.652183 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:53.652159    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8rhkt_e59810c5-e353-4644-b26c-404664df8c06/kube-multus-additional-cni-plugins/0.log"
Jul 09 03:19:53.672360 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:53.672308    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8rhkt_e59810c5-e353-4644-b26c-404664df8c06/egress-router-binary-copy/0.log"
Jul 09 03:19:53.691181 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:53.691163    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8rhkt_e59810c5-e353-4644-b26c-404664df8c06/cni-plugins/0.log"
Jul 09 03:19:53.709835 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:53.709817    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8rhkt_e59810c5-e353-4644-b26c-404664df8c06/bond-cni-plugin/0.log"
Jul 09 03:19:53.728398 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:53.728381    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8rhkt_e59810c5-e353-4644-b26c-404664df8c06/routeoverride-cni/0.log"
Jul 09 03:19:53.746901 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:53.746884    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8rhkt_e59810c5-e353-4644-b26c-404664df8c06/whereabouts-cni-bincopy/0.log"
Jul 09 03:19:53.765702 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:53.765681    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8rhkt_e59810c5-e353-4644-b26c-404664df8c06/whereabouts-cni/0.log"
Jul 09 03:19:54.159626 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:54.159594    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jt6w4_29ffb437-ae69-4765-85d3-5f1ebe2c0346/network-metrics-daemon/0.log"
Jul 09 03:19:54.175574 ip-10-0-129-236 kubenswrapper[2578]: I0709 03:19:54.175551    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jt6w4_29ffb437-ae69-4765-85d3-5f1ebe2c0346/kube-rbac-proxy/0.log"